Fragmentation prevention with IntelliWrite -- say hello to reality

Monday, January 4, 2010 by Joe Abusamra

                                                

You may have read recently about our favorite competitor's (Diskeeper's) latest release and its introduction of a file system filter for defragmentation called IntelliWrite.

As a major developer of enterprise and consumer software for defragmenting computers, many inquiring minds in the industry have begun to ask us what our opinion of IntelliWrite is. It's the kind of inquiry we expect, and eagerly answer. Erwin Solis, PerfectDisk product manager, went about directing our lab tests on the issue. Rather than give you just opinion, we attempt to provide solid numbers and easily reproducible proof.

So what is IntelliWrite and why does it create free space fragmentation?

The good folks at Diskeeper questioned the importance of total free space consolidation -- getting the biggest possible piece of free space -- in a recent blog post. The argument Diskeeper appears to be making is that free space consolidation can be ignored in favor of using a file system filter to eliminate file fragmentation on the fly. There are three key issues with this theory. First, if you intentionally (or unintentionally) cause free space fragmentation to build up, you'll quickly leave a volume in state where files will be forced to fragment. Secondly, in order to prevent such an issue from happening, you'll have to consolidate free space anyway and therefore consume even more resources. Lastly, by using a file system filter, if not done properly, additional overhead is introduced to accomplish something that NTFS already does on its own.

At the heart of this issue is the apparent insistence that the NTFS file system is incapable of preventing file system fragmentation on its own. Actually, NTFS does an excellent job all on its own, and does an even better job when free space is effectively consolidated.

So - how about a reproducable demonstration?

For the following test, feel free to use any sized volume, large or small. The key is to have plenty of consolidated free space. In order to be completely fair, we've used a freshly formatted 100GB volume so that neither NTFS nor Diskeeper's IntelliWrite will have an excuse for their behavior.

This is what we did:

1) Turned IntelliWrite off
2) Completed a Full Install of Office 2007 on a freshly formatted 100GB volume.
3) Analyzed the volume with Diskeeper and noted the statistics provided.

As you can imagine, we then ran the same test over again, this time with IntelliWrite turned on. Here are the results:

Using a 100GB NTFS volume, we can see a dramatic difference in how free space is consolidated:

 

 

NTFS

IntelliWrite

Free Space Fragmentation

 

 

Percent low performing free space

0%

0%

Total free space extents

2

3,995

Largest free space extent

97 GB

94 GB

Average free space extent size

49GB

25 MB

 

 

 

 

 

 

Low-Performing files percentage

 

 

% of entire volume

0%

0%

% of used space

0%

0%

 

 

 

Directory fragmentation

 

 

Total directories

182

182

Fragmented directories

28

21

Excess directory fragments

103

46

 

 

 

File fragmentation

 

 

Total files

4,486

4,494

Average file size

267 KB

267 KB

Total fragmented files

0

0

Total excess fragments

0

0

Average fragments per file

1.00

1.00

Files with performance loss

0

0

 

The data shows that, for the area of the disk where data was written, the average free space gap dropped from 49GB to just 25MB. Instead of free space being broken into just 2 fragments, Intelliwrite splits it into nearly 4,000 pieces. Notice how in this example that NTFS managed to prevent all file fragmentation on its own, without the additional resource impact associated with a 3rd-party file system filter. In this case, IntelliWrite provides ZERO benefit and instead proves to be the exact opposite of what you would consider a "green" technology. Try adding the I/O Other Bytes column to the Processes tab under Task Manager to see just how much I/O activity Diskeeper generates over time; we believe you'll find Diskeeper's resource usage to be a real eye opener. These are the results that IntelliWrite provides during the installation of a common office application; the impact for servers is even greater. Recently, a Diskeeper customer contacted us about a problem that IntelliWrite caused for them on one of their file servers; we saved a snapshot of the summary statistics provided by PerfectDisk Server:

 


As you can see, IntelliWrite was effectively addressing the file fragmentation on the volume - but at the cost of creating other performance issues. By eliminating the resource overhead associated with IntelliWrite and applying SMARTPlacement with PerfectDisk Server, performance was restored just 24 minutes later. Here's an analysis of the results:



As many users know, the PerfectDisk solution to this problem is to combine the intelligent placement of files, with the best possible consolidation of free space - so that NTFS can do the best job possible without additional resource overhead. After all, it's not just whether or not files are fragmented, but how and where they are written that also matters.

Our impression is clear. We suggest that users simply ignore the currently available file system filter - just turn it off. Even if Diskeeper is your preferred disk defrag utility, don't create free space fragmentation to the detriment of your system performance. 

Finally, regarding the issue of IntelliWrite being a "green" solution...because it creates a situation where resources are consumed in order to prevent file fragmentation - but which results in the need to consume yet more resources to clean up the free space fragmentation issues it leaves behind...there might be another color I'd use to describe it.

The bigger point here -- make sure you do your own testing and verification. As with just about all software, results may vary. But do your own checking -- just because someone says something is so, doesn't necessarily mean that something is so, right? I think we learned that lesson time after time in 2009.

                                                      
 
Note: Diskeeper is a registered trademark and IntelliWrite is a trademark of Diskeeper Corporation.

Share |

Comments for Fragmentation prevention with IntelliWrite -- say hello to reality

Tuesday, January 5, 2010 by Lazar:
You guys are totally right! PerfectDisk 10 is way better than Diskeeper 2010 and much cheaper. I hope PerfectDisk 11 beta starts soon.
Tuesday, January 5, 2010 by Unknownxzq:
Hi, I'm a software tester and Omg I tested diskeeper and my pc went slow. Intelliwrtite is a scam and also there Ifaast. Diskeeper has also many bugs! It every time it tells me that it isnt automated defraged, but when i look at the options it says it's automated! After uninstalling this misleading software my pc had more response time! Raxco will always be the best defrag for me, it's cheap and easy. I have used many defrags but have to say perfectdisk does what it promises a perfect job!! Thanx for this amazing software and keep it simple!!
Tuesday, January 5, 2010 by Az7:
We did a lot of comparative tests between PD & DK 2010, and simply DK wins ... :) Diskeeper -technically- is better than yours PD.. DK is installed in all our servers and we are happy :) Thanks :) Adrian.
Wednesday, January 6, 2010 by Lazar:
Adrian you are NOT right. You must be a Diskeeper employee, go back and create a good defrag software and stop lieing. Diskeeper is not good. PerfectDisk is better.
Wednesday, January 6, 2010 by Lazar:
Oh, i forgot. Diskeeper thinks SSD needs defrag. They must be crazy or they don't know how SSD works.
Saturday, January 9, 2010 by Firenze:
Both PerfectDisk and Diskeeper are scams, their constant updates are nothing more than cries for attention AND your money, while there are free defraggers that are 100x smarter.² Don't let those shady moneysick lamers cheat on you, the only thing they do is pluck away your money, and shorten your harddrive lifespans !
Saturday, January 9, 2010 by Firenze:
Hahaha, you thieves are so good at hiding the truth !! But it's out there, and you'll feel it where it hurts :)
Sunday, January 10, 2010 by superG:
Wow some you of guys have some hate towards PD and DK. If you don't like them don't buy or use them. Why do you even have to post here. Apparently nothing better to do I guess. I've used both PD and DK, DK right now, but I'm very interested to see the new PD beta.
Wednesday, January 13, 2010 by Anon:
I think diskeepers point is that if you can prevent fragmetnation, then free space fragmentation is not importnant or less important. I am interested to know what actual overhead the driver has. Also what is IO other bytes? is this coming from the driver or diskeeper defrag? Thanks.
Wednesday, January 13, 2010 by Joe Abusamra:
Hi Anon, Thanks for your comment. Overhead depends on the level of new file creation. I/O other bytes is the measure of defrag activity - coming not from Intelliwrite, but from Diskeeper doing actual defrag. Because Intelliwrite actually causes free space fragmentation, it means more work than would normally be required for Invisitasking to "clean up" the free space fragmentation that Intelliwrite has made on the drive. Thanks, Joe
Friday, January 15, 2010 by Microsoft Bob:
What's the point of Diskeeper or PerfectDisk in this day and age? Windows Vista and Windows 7 automatically defrags at night. Also, windows disables automatic defrag for solid state disks, as SSDs have wear leveling algorithms built-in, so they never need a defrag. Any benefits perceived by a user of those types of software are probably psychological.
Friday, January 15, 2010 by Joe Abusamra:
Hi Microsoft Bob, Thanks for your comment. Well, I appreciate that your perspective, from Microsoft, is "what's the point." You can search this blog for a few posts on some of the shortcomings of Microsoft's built-in defrag solutions, and you can visit here: http://www.perfectdisk.com/windows7defrag. Regarding SSDs, I've also written about that; free space consolidation is one aspect that helps write speed. Diskeeper has a separate solution. Thousands of people every day come to PerfectDisk and Diskeeper looking for a better solution, so we'll continue to oblige them,, as I'm sure Diskeeper will. But the built-in does suffice for some, true enough. Thanks, Joe
Friday, January 15, 2010 by mltx:
I am trying both PD 10 and DK 2010. DK 2010 is criticised to leave free space fragmented after intelliWriting. But then every time I use PD to check, PD reports the drive is optimized for peak performance even though there are some fragments on free space. If the fregments on free space is no good, why doesn't PD correct that. This makes the criticism unconvinced, it seems to me. Or PD makes the free space fregmentation well controlled?
Friday, January 15, 2010 by Joe Abusamra:
Hi mltx, Thanks for your comment. PerfectDisk provides effective free space consolidation - while PD may leave some free space fragmentation, overall it generates much larger free space chunks than any other product, which you will see from your own testing. Also, PerfectDisk, if configured for aggressive free space consolidation, will do an even better job. Thanks, Joe
Monday, January 18, 2010 by mltx:
Hi Joe, thank you for your explanation. Another thing I would like to mention is that, if I analyze first with the result suggesting a defragmentation and I accept the suggestion, then PD needs to reanalyze again, which is redundant to me. In other words, if you want to analyze and then defrag, you have to analyze twice. Maybe I miss something?
Monday, January 18, 2010 by Joe Abusamra:
Hi mltx, The drives could have changed between the analyze and selecting defrag, so that is why it is done again. Also,the analyze that is done with the defrag gathers more information than what is gathered with just the "Analyze" command, including information that is needed by PD to defrag. Thanks, Joe
Monday, January 18, 2010 by Mamay Hendrayana:
I already tried Diskeeper 2010, PerfectDisk 10 and O&O disk defragmenter 12. And the winner is PerfectDisk 10 : Less Talk but do more!. and For Diskeeper 2010 : Too much talk on the paper but do nothing!, for O&O disk defragmenter 12: Very silent but do nothing. This is the fact for you guys!
Sunday, January 31, 2010 by Eddie:
interesting how diskeeper's blog post touting Intelliwrite is CLOSED for comments: http://www.diskeeper.com/blog/post/2009/12/03/the-only-way-to-prevent-fragmentation-before-it-happens.aspx
Sunday, February 28, 2010 by aerDNA:
I'm a Diskeeper man myself but I must admit that v2010 is a major disappointment. From a technical perspective, IntelliWrite is a dubious feature but the real, underlying problem is - get a load of this - philosophical in nature: in a consumer society, commercial software developers (not just Diskeeper) are compelled to periodically introduce new features just for the sake of introducing new features, so they could advertise and sell a new product. Since there's a technical limit to significant improvements, new features are often just vapor or even setbacks. Just like beauty and cleaning products with their constant invention of new and xx% more efficient formulas - unless you're a sucker, you understand it's pure marketing. In software world, it means phony improvements, useless "cool new features" and often a tendecy to bloat (Nero is an extreme example of bloating). Ultimately, it leads to bulkier and lower-performing software. Oh, and I forgot about visual design, it is imperative to change it every once in a while so it looks like something new (imho, Diskeeper 2010 icon looks like it's 4 bpp).
Monday, March 1, 2010 by Oleg Eriksson:
It look like eweek say Diskeeper IntelliWrite works as they say http://www.eweek.com/c/a/Data-Storage/Diskeeper-2010-Keeps-Disk-Defrags-to-a-Minimum-356694/ Do you agree?
Monday, March 1, 2010 by Joe Abusamra:
Hi Oleg, Thanks for your comment. Well, I wasn't in eWeek's lab so I can't comment too much. I can say that there is no mention of the activity on the drive, number of files, size of files, etc. Importantly, there is no mention of free space fragmentation or free space consolidation in the eWeek article. I'll just say that free space is an important aspect of this whole thing, and anyone looking at it should look closely. As an aside, the article's statement about the company being responsible for the defrag shipped with Windows is incorrect. Thanks, Joe
Thursday, March 4, 2010 by Oleg Eriksson:
Thank you Mr Abrusama. I agree tests are incomplete. Though Microsoft say Diskeeper made Windows XP defrag. I found this http://support.microsoft.com/kb/314848
Thursday, March 11, 2010 by JasonDiplomat:
I'm a Diskeeper user, and I totally concur Joe's investigation. After running through a painful defrag session, including a blitzed paging file, I too have decided to turn off "Intelliwrite". It was weird watching a defrag, where lots of "blue" files were being moved and opening up free space. I have applications using fixed-size DB files that became fragmented on creation - didn't make any sense until I read this blog!
Monday, May 17, 2010 by DanS:
I just purchased PD 11 Pro. It seems to be the answer for me. I use an XP box with a 1.5tb HD as a backup solution to a Security Server we have. I copy 25 files every 5 hours or so, (96 each day), each being approx 200mb using RichCopy. The files get broken into 2000 pieces, that's not an exaggeration. I'm able to schedule PD four times each day, once after each copy, to do a Consolidate Free Space. I like what I’m seeing so far. What I don’t know yet is if PD will be able to keep up with the fragmentation. Should be interesting.
Monday, May 17, 2010 by Joe Abusamra:
Thanks for the comment, Dan. Will be interested to see how PD keeps up!
Saturday, May 22, 2010 by Strychnine Niño:
I've Been using Diskeeper for years, up until now... i know every software strives to be the best and give best results to each people who are using it... i want to know who is really the best this 2010 Not just by how many the users are, but thoroughly in results, performance and everything.... i will still heed in your comments... please fill me up.
Saturday, May 22, 2010 by Strychnine Niño:
I've Been using Diskeeper for years, up until now... i know every software strives to be the best and give best results to each people who are using it... i want to know who is really the best this 2010 Not just by how many the users are, but thoroughly in results, performance and everything.... i will still heed in your comments... please fill me up.

Leave a comment





Captcha