Knowledge
- Identify appropriate BIOS and firmware setting requirements for optimal ESX/ESXi Host performance
- Identify appropriate ESX driver revisions required for optimal ESX/ESXi Host performance
- Recall where to locate information resources to verify compliance with VMware and third party vendor best practices
Skills and Abilities
- Tune ESX/ESXi Host and Virtual Machine memory configurations
- Tune ESX/ESXi Host and Virtual Machine networking configurations
- Tune ESX/ESXi Host and Virtual Machine CPU configurations
- Tune ESX/ESXi Host and Virtual Machine storage configurations
- Configure and apply advanced ESX/ESXi Host attributes
- Configure and apply advanced Virtual Machine attributes
- Tune and optimize NUMA controls
Tools
- vSphere Resource Management Guide
- vSphere Command-Line Interface Installation and Scripting Guide
- Performance Troubleshooting for VMware vSphere 4
- Product Documentation
- vSphere Client
- Performance Graphs
- vSphere CLI
- vicfg-*
- resxtop/esxtop
- vscsiStats
Notes
Identify appropriate BIOS and firmware setting requirements for optimal ESX/ESXi Host performance
Some of this will vary system to system so this is going to be one where you play around a bit and get outside of your comfort zone and using the type of system you usually do.
Hyperthreading
- ESX(i) will use hyperthreading by default, but it may need to be enabled in the bios.
- Cannot enable hyperthreading on a system with great then 32 physical cores because of the logical limit of 64 CPUs
- http://www.dabcc.com/article.aspx?id=9482
Enable Power Management
- Enable power management or Demand-Based Switching (DBS) in the bios.
- You can then configure the CPU power management policy in the advanced host attribute, Power.CpuPolicy
- You will set this to either static (default) or dynamic. In dynamic the VMkernel optimizes each CPU’s frequency to match demand. When CPU demands increase the policy ensures that CPU frequencies also increase.
- With static the VMkernel can detect power management feature available on the host but doesn’t actively use them unless requested to do so by the BIOS.
Identify appropriate ESX driver revisions required for optimal ESX/ESXi Host performance
- Same as the above, this will vary implementation to implementation. Be aware of the memory requirements for drivers.
- Some drivers need 40MB, which almost doubles base system memory.
- An ESXi host uses additional system memory for management agents that run in the service console of an ESX host.
Recall where to locate information resources to verify compliance with VMware and third party vendor best practices
http://www.vmware.com/resources/compatibility/search.php?ie=UTF-8&q=vmware%20hcl
Tune ESX/ESXi Host and Virtual Machine memory configurations
For these tuning sections I am going to recommend referring to the troubleshooting sections I’ve previously covered as well as the links at the bottom of these notes.
Tune ESX/ESXi Host and Virtual Machine networking configurations
For these tuning sections I am going to recommend referring to the troubleshooting sections I’ve previously covered as well as the links at the bottom of these notes.
Tune ESX/ESXi Host and Virtual Machine CPU configurations
For these tuning sections I am going to recommend referring to the troubleshooting sections I’ve previously covered as well as the links at the bottom of these notes.
Tune ESX/ESXi Host and Virtual Machine storage configurations
For these tuning sections I am going to recommend referring to the troubleshooting sections I’ve previously covered as well as the links at the bottom of these notes.
http://www.vfail.net/2010/08/18/objective-6-4-troubleshoot-storage-performance-and-connectivity/
Configure and apply advanced ESX/ESXi Host attributes
From pg. 97-99 of the vSphere Resource Management Guide
The process is easy but you will need to be familiar with a lot of different advanced settings and the pages above contain those settings.
- In the vSphere Client inventory panel, select the host to customize.
- Click the Configuration tab.
- In the Software menu, click Advanced Settings.
- In the Advanced Settings dialog box select the appropriate item (for example, CPU or Memory), and scroll in the right panel to find and change the attribute.
Configure and apply advanced Virtual Machine attributes
pg. 99 of the vSphere Resource Management Guide
- Select the virtual machine in the vSphere Client inventory panel, &select Edit Settings from right click menu.
- Click Options and click Advanced > General.
- Click the Configuration Parameters button.
- In the dialog box that appears, click Add Row to enter a new parameter and its value.
Advanced Virtual Machine Attributes
- sched.mem.maxmemctl Maximum amount of memory reclaimed from the selected virtual machine by ballooning, in megabytes (MB). If the ESX/ESXi host needs to reclaim additional memory, it is forced to swap. Swapping is less desirable than ballooning.
- sched.mem.pshare.enable Enables memory sharing for a selected virtual machine. This boolean value defaults to True. If you set it to False for a virtual machine, this turns off memory sharing.
- sched.swap.persist Specifies whether the virtual machine’s swap files should persist or be deleted when the virtual machine is powered off. By default, the system creates the swap file for a virtual machine when the virtual machine is powered on, and deletes the swap file when the virtual machine is powered off.
- sched.swap.dir VMFS directory location of the virtual machine’s swap file. Defaults to the virtual machine’s working directory, that is, the VMFS directory that contains its configuration file. This directory must remain on a host that is accessible to the virtual machine. If you move the virtual machine (or any clones created from it), you might need to reset this attribute.
Tune and optimize NUMA controls
Pg. 73-77 of the vSphere Resource Management Guide
- The VMkernel.Boot.sharePerNode option controls whether memory pages can be shared (de-duplicated) only within a single NUMA node or across multiple NUMA nodes.
- VMkernel.Boot.sharePerNode is turned on by default, and identical pages are shared only within the same NUMA node. This improves memory locality, because all accesses to shared pages use local memory.
- In most situations the ESX(i) host’s automatic NUMA optimizations will result in good performance, however you may need to tune this.
- You can do this through controlling memory and processor placement via CPU and Memory Affinity.
Other Relevant Reading Related To This Section
- http://www.virtualinsanity.com/index.php/2010/03/29/performance-troubleshooting-vmware-vSphere-network/
- http://www.virtualinsanity.com/index.php/2010/03/16/performance-troubleshooting-vmware-vSphere-storage/
- http://www.virtualinsanity.com/index.php/2010/02/19/performance-troubleshooting-vmware-vSphere-memory/
- http://www.virtualinsanity.com/index.php/2010/02/15/performance-troubleshooting-vmware-vSphere-cpu/