ESXi 4.1 remote log collection with VMware vMA 4.1

When I first decided to blog about this, I started doing some Googling for resources. I ran across an excellent post by Simon Long that talked about using vMA’s vilogger capabilities for remote logging, but his blog post was written with 4.0 in mind, and didn’t line up exactly with how things are done with vMA 4.1. My goal here is not to reinvent the wheel with screenshots galore, but merely to document the basic steps and differences in the setup for vMA 4.1.

Here’s a short outline of the basic steps:

• Set NTP to UTC on vMA and enable NTP
• Add additional storage to vMA and mount it to a directory within vMA
• Change the vilogger settings to point collected logs to the newly mounted directory (disk)
• Add servers to vMA using fastpass
• Enable log collection
• Claim your title as "most awesomest VMware admin, ever"

Set Time & Enable NTP

Since ESXi uses UTC for its time stamping (and there’s no way to change it), it’s a good idea to set vMA’s time zone to UTC (PDT is default on vMA).

• sudo rm /etc/localtime
• sudo ln -s /usr/share/zoneinfo/UTC /etc/localtime
• sudo chkconfig –level 2345 ntpd on
• Edit /etc/ntp.conf and replace default ntp.org servers with your internal NTP servers where applicable (sudo nano /etc/ntp.conf)
• Reboot or start ntpd
• Verify settings with ntpq -p

Add Additional Storage to vMA

Shutdown the VM, and add an additional hard disk to the VM. I chose to go with 20GB additional. Power the VM back on.

• Format the new disk: sudo fdisk /dev/sdb
• Press n to create a new partition
• Press p to make the new partition primary
• Press 1 to make the partition number 1
• Use the default on the First and Last cylinders
• Press p to verify the partition table
• Press w to write the partition table
• When its complete, format the new disk: sudo mkfs -t ext3 /dev/sdb1
• Edit /etc/fstab: sudo nano /etc/fstab
• Enter the following line (use Tab to line up the entries):
  /dev/sdb1    /var/log/syslog    ext3   defaults, auto     1 2
• Press Ctrl+X and then Y to save and close
• Make the new directory: sudo mkdir /var/log/syslog
• Change the owner of the new directory:  sudo chown vi-admin:root /var/log/syslog
• Mount everything in /etc/fstab with sudo mount -a. You should receive no mount errors, and executing sudo df -h should list the /dev/sdb1 as being mounted to /var/log/syslog

Change vilogger Settings

• Edit vilogger’s config file: sudo nano /etc/vmware/vMA/vMA.conf
• Change the location entries (three of them) to <location>/var/log/syslog</location>
• Start (or restart) the vilogger daemon: sudo service vmware-vilogd start

Enable Remote Logging

• Add your ESXi host using fastpass authentication: sudo vifp addserver <FQDN of server>
• Enter the root password for the server when prompted
• Verify the server has been added to vifp: vifp listservers
• Enable vilogging: vilogger enable –server <FQDN of server> –numrotation 20 –maxfilesize 10 –collectionperiod 300 (these settings can be changed; learn more at vilogger –help)

Tips

• You can add servers in bulk with a sample Perl script located at /opt/vmware/vma/samples/perl/builkAddServers.pl (Read the README for more information)
• According to the documentation, vMA has been tested with over 100 targets under normal load conditions. I recommend testing different log collection intervals and host counts if you are near that number.
• Be sure to size appropriately. ESXi logging collects 3 logs (vpxa.log, hostd.log, and messages.log). Rough math is number of logs X number of hosts X maxrotations X maxfilesizes = total MB for logs. You’ll need slightly more than that for vilogger’s logs (depending on where you placed those when you edited vMA.conf).

Resources

• http://communities.vmware.com/docs/DOC-10501.pdf
• http://www.vmworld.com/docs/DOC-5128
• vMA 4.1 Documentation
• Simon Long’s article on vMA
• William Lam’s article on vMA Perl scripts