I would like to introduce myself. My name is Bob Nolan and I am the president of Raxco Software, developers of the PerfectDisk line of disk defragmentation software. The internet has provided a fantastic forum for anyone to post their opinions on most any topic. When it comes to disk defragmentation and the Windows file systems, some of this information is correct and some is not. In this blog, I am going to try to provide accurate information about disk defragmentation, PC performance, disk space management and more.
Raxco has been working with VMware the past several months to assess the impact of Windows guest defragmentation on VMware performance. The most recent testing used VMware's vscsiStats utility to capture metrics. The vscsiStats utility sits between the VMware kernel and the Monitor. It captures every IO request through the storage stack and sorts them into various buckets that can be used for performance analysis. We ran our test concurrently using five virtual machines on an ESX 4.0 cluster.
The goal of the test was to determine if Windows guest defragmentation improves virtualization performance. The test was fairly simple; we created a fragmented disk with ISO images and other files and made a copy for each of the five virtual machines. We installed MS Office and MS SQL on each of the VMDKs and measured the results with vscsiStats. After the data collection was done on the fragmented disks, we used PerfectDisk to defragment the files and free space on all of the disks and repeated the tests using the defragmented disks.
The results were pretty impressive on several fronts. First of all, one of the main sources of virtualization overhead occurs when IO have to traverse the virtual storage stack. Our testing showed that when the Windows guests were defragmented with PerfectDisk the total number of IO was reduced by 36.5% and write IO was reduced by 44.6% after the MS SQL installation. This means defragmentation was able to reduce the total IO load by over 1/3 for all IO and it reduced write IO by almost half. How was defragmentation able to do this?
The answer is that while file defragmentation improves file read time, free space consolidation improves file write time by reducing the number of physical accesses needed to write the file. If the guest file system can find sufficient contiguous free space, the likelihood of the file being written in one piece improves. Contiguous free space also means you do bigger writes. The vscsiStats bear this out. The PerfectDisk disk created 11.9 times as many IO greater than 524K, which is the largest vscsiStats bucket size. The vscsiStats histogram illustrates this difference on the far right column.
Disk IO latency was also affected by the file defragmentation and free space consolidation of the Windows guests. As noted above, the Windows file system was able to perform fewer and larger IO when the disk is defragmented and the free space is aggregated in a large chunk. It logically follows that these IO will complete in less time and the vscsiStats again support this assumption. The utility sorts IO into buckets ranging from .001ms to >100ms. Since the average disk drive access time is about 15ms, we viewed all IO taking more than 15ms as a slow IO. The vscsiStats show the defragmented disk reduced the total number of IO taking longer than 15ms by 48.7%.
The final metric that showed improvement due to guest system defragmentation was sequential IO. The vscsiStats utility measures the distance in logical blocks between seeks. This histogram shows that the defragmented disk increased the number of IO that were only one block away (center columns) and decreased the number of IO that were 1000 to 500,000 logical blocks away (left and rightmost columns).
In summary, it appears Windows guest file defragmentation and free space consolidation will:
- Reduce VMware overhead by reducing the total number of IO that traverse the virtual storage stack
- Improve system throughput by producing larger IO
- Reduce system latency by reducing total IO and generating larger IO
- Improve virtualization performance by increasing sequential IO
- Reduce the demand for host CPU, memory and IO resources
- Potentially create a situation where the host can support one or more additional guests due to the resources conserved
To access a complete copy of the white paper detailing the test procedures and results go to www.perfectdisk.com/user_data/white_papers/vmware_multi_test_new.pdf
A week ago I had the opportunity to speak at the VirtG Deep Dive in Waltham, MA. The Deep Dive is a day long event focusing on Microsoft's Hyper-V virtualization solution. My topic was the impact of file and free space fragmentation in a virtual environment.
The effects of fragmentation in a Hyper-V environment can be more profound than they are on a stand-alone physical server. Hyper-V performance suffers because there is fragmentation on the virtual guests and fragmentation on the host. This can result in I/O contention between the virtual guests, and between the virtual guests and the host. There is a finite amount amount of resources available and excessive I/O in the virtual guests or on the host due to file and free space fragmentation can stress the system. This is why it is necessary to defrag Hyper-V on both the virtual guest and the host ends of the system.
Hyper-V, or any virtualization platform for that matter, takes steps to provide good performance. However, if a virtual guest generates unecessary I/O, the impact will be felt by the rest of the system. Defrag reduces I/O loads and minimizes I/O contention between virtual guests and the host. Hyper-V performance monitoring indicates that defragmenting the guest and the host reduces the total number of I/Os on the whole system. Defrag, along with free space consolidation, should be part of of any Hyper-V performance-tuning efforts.
My presentation at the Deep Dive was well received and there were a lot of questions afterward. Several attendees indicated they thought fragmentation could be part of their Hyper-V performance issues. I told them to defrag Hyper-V on a regular basis to get the most from their virtualization investment. Defragmenting host and guests should be part of any Hyper-V performance tuning best practices. Of course, I had to tell them the best defragmentation came from PerfectDisk.
The effects of fragmentation in a Hyper-V environment can be more profound than they are on a stand-alone physical server. Hyper-V performance suffers because there is fragmentation on the virtual guests and fragmentation on the host. This can result in I/O contention between the virtual guests, and between the virtual guests and the host. There is a finite amount amount of resources available and excessive I/O in the virtual guests or on the host due to file and free space fragmentation can stress the system. This is why it is necessary to defrag Hyper-V on both the virtual guest and the host ends of the system.
Hyper-V, or any virtualization platform for that matter, takes steps to provide good performance. However, if a virtual guest generates unecessary I/O, the impact will be felt by the rest of the system. Defrag reduces I/O loads and minimizes I/O contention between virtual guests and the host. Hyper-V performance monitoring indicates that defragmenting the guest and the host reduces the total number of I/Os on the whole system. Defrag, along with free space consolidation, should be part of of any Hyper-V performance-tuning efforts.
My presentation at the Deep Dive was well received and there were a lot of questions afterward. Several attendees indicated they thought fragmentation could be part of their Hyper-V performance issues. I told them to defrag Hyper-V on a regular basis to get the most from their virtualization investment. Defragmenting host and guests should be part of any Hyper-V performance tuning best practices. Of course, I had to tell them the best defragmentation came from PerfectDisk.
There have been a number of blogs and articles written in recent months on the topic of best defrag software. I think that just begs the question, what would you want in your best defrag software?
At a minimum, you would want any defrag software to defragment all of your data files. The Windows 7 defragmenter and the free defrag utilities do a decent job in this regard, but the disks still re-fragment quickly. File defragmentation is just half the job, to truly hinder file fragmentation you need a defrag tool that also consolidates the maximum amount of free space. If the file system can find contiguous free space it will write files in one piece.
So, if you start to really look at what constitutes the best defrag software you want something that defragments all your files and consolidates the maximum amount of free space. But is there more to look for in a disk defrag program?
The location of the files can also make a difference. Files that don't change often can be defragged and placed together. The next time the defragger runs these files don't have to be moved. The same goes for frequently changing files, they can be located near the free space so they can grow in the fewest number of pieces. Boot files can be placed next to the Master Boot Record for faster system boot ups, and you can defrag system files, the ones Windows 7 doesn't touch, to get them out of the way so your on line defrag is faster.
Combine all of this with flexible scheduling, automatic defragmentation, a duplicate file finder, virtual system support and ease of use and you have something that really qualifies as the best defragmentation software. This is why PerfectDisk 10 is the choice of thousands. These users know that the best defragmentation software does the whole job.
I was recently invited to speak at a VMware User Group meeting about the impact of file and free space fragmentation in a virtual environment. Before I started the presentation I asked the audience how many had ever defragmented a physical server and about 30% raised their hand. Then I asked how many had ever defragmented a virtual server and about 10% raised their hand. This is a pretty typical response.
Systems administrators running virtual machines seem to overlook the fact that each virtual machine is running its own version of Windows Server and within that machine NTFS is busy fragmenting files and free space. While fragmentation is a performance-stealing problem on physical servers, the problem is compounded on virtual machines where each virtual instance is competing for a finite amount of resources. This leads to resource contention between Windows guests and, with Hyper-V, contention between the Windows guests and the host. It is essential to keep the files defragmented and the free space consolidated on Windows guests to maximize performance and minimize resource contention in a virtual world.
After my presentation one attendee told me his company was having performance issues with several of their virtual servers. They were looking at upgrading the hardware and bringing in a consultant. He knew these virtual machines were frequently updating files, the kind of activity that produces fragmentation, but he never thought of fragmentation as the problem. After sitting through my presentation he was convinced fragmentation was the culprit and we just saved them a lot of unnecessary expense. In complex technical environments it is easy to overlook the obvious when looking for a solution. Sometimes you just need to defrag a computer to get a lot more out of it, and that includes VM's.
Powered by Compendium Blogware