It’s not surprising that almost all VMware performance issues are I/O related. Think about it, you have lots of VMs sending all their I/O to the same storage array so the LUN queue becomes the bottleneck. VMware has long asserted that having fewer IOPs is better than having lots of IOPs, but exactly how are you supposed to reduce your total IOPs?
Before we answer this let’s define what an IOP is. From VMware’s perspective an IOP is a SCSI command originating in the Windows guest that is processed by the hypervisor and the storage controller. The number of IOPs produced is directly related to the condition of your files as they are seen by the Windows file system, NTFS. The way NTFS views a file is different from the way the storage controller views it. Here is what is going on “under the covers” in NTFS.
In the vSphere documentation, under the heading Disk I/O Performance Enhancement Advice, the second recommendation is to “Defragment the file systems on all guests”. https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.monitoring.doc/GUID-174326D5-238B-48CA-B030-02009E388523.html VMware recognizes that guest defragmentation reduces the SCSI workload and improves overall system performance.
When a user “saves” a file in Windows a file record is created in the Master File Table (MFT) and then a special file on the VMDK ($Bitmap) is queried and offers up chunks of logical address space to accommodate the file. Each chunk of space is recorded in the Extent List which is a structure in the file record in the MFT. Each entry in the Extent List identifies where the chunk starts and how many clusters it includes. If the file is saved in more than one entry, then it is a fragmented file as far as NTFS is concerned, even though nothing has been written to a disk yet.
Each entry in the Extent List becomes a SCSI command that is processed by the hypervisor on its way to the storage controller. The controller maps the logical address information to a physical address(es) on the array. A file saved in 100 chunks of address space will need 100 SCSI commands to move it to the controller and 100 physical disk I/O would be the absolute minimum number of physical disk I/O needed to write the file to the array ( likely more due to disk striping). In short, every file fragment becomes a SCSI command (IOP).
Defragmentation works at the NTFS level, it consolidates the chunks of address space in each files Extent List so the file can ideally be accessed in a single, larger SCSI command. This reduction in the SCSI (IOP) workload eliminates LUN queue contention which in turn reduces the number of physical I/O requests going to the array. When fewer physical I/O are needed, disk latency can improve by 50-80%.
PerfectDisk vSphere is specifically designed to operate in a VMware environment. On average PerfectDisk vSphere reduces total IOPs by 25-40%. Guest defragmentation with PerfectDisk vSphere leverages your VMware investment by making every VM runs at its maximum performance level.