The issue has now been resolved in a new patch released a few days ago:
VMware KB: VMware ESXi 5.1, Patch ESXi510-201402401-BG: Updates esx-base
This patch updates the esx-base VIB to resolve the issue where virtual machines using the E1000/E1000E adapter might lose network connectivity when the networking heap memory, netGPheap, reaches the maximum limit. In the vmkernel.log file, you see entries similar to:
WARNING: Heap: 2677: Heap netGPHeap already at its maximum size. Cannot expand.
WARNING: Heap: 3058: Heap_Align(netGPHeap, 35156/35156 bytes, 8 align) failed. caller: 0x41800915066c
WARNING: E1000: 8817: failed to enable port 0x200002f on DvsPortset-0: Out of memory
WARNING: Net: vm 367615: 4454: cannot enable port 0x200002f: Out of memory
For more information about symptoms and resolution, see KB 2072694.
