Eric Sloof blogged on this, and I thought I would as well.
since we updated our ESX hosts from ESX 3.5 Update 2 to ESX 3.5 Update 3 (+ Dec 2008 patches) we experience an abnormally high RAM usage of the hostd management process on the hosts.
The symptoms are that hostd hits its default hard memory limit of 200 MB and shuts itself down. This is indicated by the following error message in var/log/vmware/hostd.log:
[2009-01-06 06:25:47.707 ‘Memory checker’ 19123120 error] Current value 207184 exceeds hard limit 204800. Shutting down process.
The service is automatically restarted, but as a consequence the VirtualCenter agent also restarts and the host will become temporarily unresponsive in VirtualCenter (shown as "not responding" or disconnected").
We increased the hard limit to 250 MB, but within several hours this limit was also reached on some hosts and the problem re-appeared. So, I suspect that there is ome kind of memory leak in hostd, and we need to find its cause to finally solve the problem.
If anyone has any idea’s on this please post a reply on the VMware Community forums. Or comment / post them on the Blog / blog forums and I will link them on the VMware Community Forums.