Recently, I had the opportunity to attend VMworld 2011 in Las Vegas. The big buzz at the show was about VMware vSphere5 which is all about enabling cloud-scale operations. About 250 vendors presented at the Solutions Exchange including Citrix, a VMware rival, and a host of other vendors offering competing virtual desktop solutions.
Backup, replication and deduplication offerings were abundant among many vendors. There were also a number of companies touting monitoring solutions that show you where the bottlenecks are in your virtual environment. I found this interesting for two reasons:
- It demonstrates virtualization does not cure all ills. Once you move to a virtual environment the same types of issues arise that are evident on physical systems. System throughput, queue clogging, and disk latency can all have an adverse effect on virtual system performance.
- These problems are now commonplace enough for a market to emerge to address the issues.
In my three days at the show I had the chance to talk to lots of vendors but more interesting were chats I had with ordinary attendees who approached me at lunch and during breaks. Most of these folks were system administrators and the chats were informative. Its clear there is confusion out there on the applicability of defragmentation to virtual machines; I call these the defrag myths. In many ways they are the same myths we have long dealt with in the physical server space with a new virtual spin. Here are some of the more common misconceptions I found myself explaining in many discussions:
Myth #1 – You Don’t Need to Defrag Virtual Machines
File and free space fragmentation are well-documented problems with Windows. Microsoft provides a defragmentation tool with Windows for a reason. Over the years Raxco has sold PerfectDisk to tens of thousands of companies to address fragmentation-related performance issues on physical servers where a single box runs a single instance of Windows Server. When you virtualize, Windows Server behaves the same way it does on a physical server; it fragments the vmdk or vhd. Several instances of Windows Server on an ESXi host all compete for finite CPU, memory and disk resources. Fragmentation increases the number of SCSI commands across the storage stack which increase hypervisor overhead. More SCSI commands increases physical disk I/O, degrades disk latency and the result is I/O contention. Guest defragmentation goes a long way towards fixing this. I spoke with one guy who uses PerfectDisk on his physical servers, but never thought of using it on his virtual disks. He said this would change after he returned from the show.
Defrag software, just like NTFS, has no idea of the underlying storage a system uses. It can’t tell the difference between SCSI, IDE, RAID or a SAN. Defragmentation occurs on the Windows guests to optimize how NTFS “sees” the disk, not how the disk controller sees it. When a file is accessed, fragmented or defragmented, NTFS sends the information it has on the file to the disk or SAN controller. The disk/SAN controller maps the information from NTFS to physical blocks on the disk. The defragmentation software has no influence or control over where any data ultimately winds up on the disk(s). Physical disk location is determined solely by the disk/SAN controller. Defragmentation software cannot undo LUNS or otherwise mess up SAN structures.
Myth #3 – You Shouldn’t Defrag if You Have Dedupe
This one confuses a lot of people and it is a myth perpetuated mostly by deduplication vendors. Defragmentation and deduplication are both system maintenance activities aimed at better performance and space recovery respectively; they are not mutually exclusive. Raxco has a large customer that uses our Enterprise Console to monitor their systems. When a condition arises that requires defragmentation they run PerfectDisk and follow it with a deduplication getting the best from both technologies.
Myth #4 – Hardware is the Answer
Several of the attendees I spoke with indicated they had experienced I/O contention issues. In order to address the problem, they purchased additional host systems to spread out their VMs, upgraded fiber channel and increased memory. In a virtual environment all the work is still done on the guest system which is usually Windows Server. As noted in Myth #1, Windows file system behavior on a busy VM increases the demand for CPU, memory and especially disk resources. If you keep the Windows guest in good shape you can reduce the resource demand. We did some testing and used VMware’s vscsiStats utility to collect the data. We compared the statistics on two sets of identical disks where one set was the baseline and the other set was optimized with PerfectDisk. The vscsiStats showed the PerfectDisk disks delivered a:
- 28% reduction in total I/O across the storage stack
- 49% reduction in disk latency (reduction in I/O taking >30ms to complete)
- 58% improvement in sequential I/O
- 12x increase in larger I/O
- 28% improvement in throughput
Instead of spending tens of thousands of dollars for new hardware that doesn’t fix the problem, PerfectDisk can keep virtual disks running in optimal condition and eliminate sources of I/O contention.
VMworld 2011 was an interesting event with a lot of technology on display. It appears an entire industry is sprouting up around the virtualization movement. There is no doubt virtualization will bring many companies considerable savings through lower utility costs, smaller data centers and lower hardware costs. The virtualization promise is best realized when administrators properly manage virtualized guest systems to reduce resource demand.
CEO, Raxco Software