Jason Boche has taken the time to outline Idle Memory Tax in VMware ESX.
Jason, I wanted to thank you for taking the time on this, and I am hoping you do not mind me quoting your Article.
“
Memory over commit is a money/infrastructure saving feature that fits perfectly within the theme of two of virtualization’s core concepts: doing more with less hardware, and helping save the environment with greenness. While Microsoft Hyper-V offers no memory over commit or page sharing technologies, VMware has understood the value in these technologies long before VI3. I’ve mentioned this before – if you haven’t read it yet, take a look at Carl Waldspurger’s 2002 white paper on Memory Resource management in VMware ESX Server.
One of VMware’s memory over commit technologies is called Idle Memory Tax. IMT basically allows the VMKernel to reclaim unused guest VM memory by assigning a higher “cost value” to unused allocated shares. The last piece of that sentence is key – did you catch it? This mechanism is tied to shares. When do shares come into play? When there is contention for physical host RAM allocated to the VMs. Or in short, when physical RAM on the ESX host has been over committed – we’ve granted more RAM to guest VMs than we actually have on the ESX host to cover at one time. When this happens, there is contention or a battle for who actually gets the physical RAM. Share values are what determine this. I don’t want to get too far off track here as this discussion is specifically on Idle Memory Tax, but shares are the foundation so they are important to understand.
Back to Idle Memory Tax. Quite simply it’s a mechanism to take idle/unused memory from guest VMs that are hogging it in order to give that memory to another VM where it’s more badly needed. Sort of like Robin Hood for VI. By default this is performed using VMware’s balloon driver which is the more optimal of the two available methods. Out of the box, the amount of idle memory that will be reclaimed is 75% as configured by Mem.IdleTax under advanced host configuration. The VMKernel polls for idle memory in guest VMs every 60 seconds. This interval was doubled from ESX2.x where the polling period was every 30 seconds.
Here’s a working example of the scenario:
- Two guest VMs live on an ESX/ESXi host with 8GB RAM
- Each VM is assigned 8GB RAM and 8,192 shares. Discounting memory overhead, content based page sharing, and COS memory usage, we’ve effectively over committed our memory by 100%
- VM1 is running IIS using only 1GB RAM
- VM2 is running SQL and is request the use of all 8GB RAM
- Idle Memory Tax allows the VMKernel to “borrow” 75% of the 7GB of allocated but unused RAM from VM1 and give it to VM2. 25% of the unused allocated RAM will be left for the VM as a cushion for requests for additional memory before other memory over commit technologies kick in
Here are the values under ESX host advanced configuration that we can tweak to modify the default behavior of Idle Memory Tax:
- Mem.IdleTax – default: 75, range: 0 to 99, specifies the percent of idle memory that may be reclaimed by the tax
- Mem.SamplePeriod – default: 60 in ESX3.x 30 in ESX2.x, range: 0 to 180, specifies the polling interval in seconds at which the VMKernel will scan for idle memory
- Mem.IdleTaxType – default: 1 (variable), range: 0 (flat – use paging mechanism) to 1 (variable – use the balloon driver), specifies the method at which the VMKernel will reclaim idle memory. It is highly recommended to leave this at 1 to use the balloon driver as paging is more detrimental to the performance of the VM
VMware recommends that changes to Idle Memory Tax are not necessary, or even appropriate. If you get into the situation where Idle Memory Tax comes into play, you need to question the VMs that have large quantities of allocated but idle memory. Rather than allocating more memory to the VM than it needs, thus wrongly inflating its share value, consider reducing the allocated amounts of RAM to those VMs.
“
I thought this was very interesting, I my self have never played with this setting.
I did have some questions on performance on both the VMware Host side, and VMware Guest side. Do either have a performance hit when ESX is using Idle Memory Tax to move that memory.
I could see large uses for this, if you have a system that only uses large amounts of memory at certain times of the day.
In fact, would you want to design your systems to use this ( by forcing low ram ), if you have certain systems that run at night, when other systems are idle ( with more ram) that normally are only used during the day?
Source Post http://www.boche.net/blog/index.php/2009/01/29/idle-memory-tax/
