Whatever Happened To ... The Defragmentation App?

In this installment of Whatever Happened To ... ?, I'll be talking about something that I touched on before in the past -- the way Windows Vista's defragmentation application has been simplified rather drastically.  There are some possible ameliorations, however.

[Note: this entry contains some previously published information in a new form.  If you've read the earlier piece I did on this subject, I encourage you to revisit this one.]

1. What's the Defragmentation App?

In Windows XP, the Defrag app was either the command-line or GUI application that allowed you to defragment the computer on demand.  It was also possible to set up the Task Scheduler to run the command-line version of Defrag on a regular basis.

2. What Happened to the Defragmentation App?

In Windows Vista, the Defrag GUI was replaced with something completely different:

You'll notice that one of the things now missing is any kind of statistics or progress bar.  This caused a great deal of annoyance among experts, who felt they were being deprived of valuable information.  What's the point of having a tool like this if we can't even see the progress meter?

Here is why this issue is not as obvious as it might seem:

1. Automatic scheduling

One of the things that has been added is an automatically scheduled defragmentation job that runs once a week.  In other words, a user who runs Vista and knows nothing about defragmentation will have their drive defragmented for them in the background during periods of low activity.  It means one less thing for everyone, expert and amateur alike to worry about.

2. Generic defragmentation statistics are not always useful

Second, the more I learn about fragmentation on a drive, the more I realize there is no one convenient metric that you can use to describe how badly a drive is fragmented.  You could talk about individual file fragments, but what if they're for files that are rarely accessed?  Or free space fragmentation, but what if the free space "fragments" are so large (50MB or more) that it's not going to create an issue for any new file written in it?

I guess it might be possible to come up with some kind of global "fragmentation index" -- like the way many of the newer anti-malware programs have some kind of "threat index" or "system protection index", but in the end I'm not sure that gives us any more useful information than we had before.  If a drive is fragmented badly enough that it's impacting performance, then we tend to take a fire-and-forget approach to the whole thing: we defrag the drive and move on to doing more serious work.

3. Defrag progress statistics can also be misleading

But what about the progress display?  Here's a list of the reasons for eliminating the display (and the lack of any progress data), straight from the horse's mouth:

The new interface seems “dumbed down.” Why remove all the detail?
...one of the biggest and consistent complaints we had from users (broad sample here from home users to experienced IT Pros) in the past was that a vast majority of them had no idea what the detailed fragmentation statistics they saw meant. The Windows XP graphical view also had some limitations and inaccuracies that prevented it from being included in Windows Vista. If you really want to keep a close eye on fragmentation, I’d recommend using the command-line tool Defrag.exe. [Emphasis mine]

Why was the defrag progress indicator removed?
Part of the problem with the Windows XP defrag tool was that percent complete was not accurate or meaningful. Depending on the phase of defrag, 1% of progress could take from several seconds to minutes, which made the progress indicator highly unreliable. The difficulty here is that since defrag is a multi-pass process (multiple iterations of file defragmentation and free space consolidation) there is no way to accurately predict when defrag will complete since the number of loop iterations and how long each takes are highly dependent on the layout of the files on the volume, the level of file and free space fragmentation, and the other system activity. While I agree that having no progress is bad, misleading progress I believe is worse. Also, the idea behind the new automated defrag is that users will not have to think about it not worry about the progress it is making. With defrag running regularly, the system will be close to optimal levels of fragmentation, and subsequent defrag runs should not take long. [Emphasis mine]

I should note that there has been a significant change to the way the defragmenter "sees" fragments on NTFS volumes.  Any given file fragment that is larger than 64MB is not consolidated by default -- it's left as-is.  This is because a fragment that large is probably something that's read sequentially anyway, so it probably won't appreciably improve the performance of the file if you forcibly collect all of its fragments and line them up.  (A file that large also doesn't tend to get read all the way through very often.)

3. What's To Be Done?

1. Leave it alone

This sounds like blunt, almost know-nothing-ish advice, I know.  But from what I have seen it's better to simply let Vista deal with fragmentation in its own way and at its own pace than to try and micromanage the whole thing (and with questionable performance returns).  In the long run, your time will be better spent on other things.

2. Use the CLI defrag

If you must defrag manually, use the CLI defrag command.  For a really aggressive cleanup, run it once a month with the following options, preferably before you go to bed:

defrag c: -v -w

This produces a verbose defragmentation report and forces a full consolidation of files.  It's actually useful to do this if you just upgraded from an earlier version of Windows and the drive is a mess of fractured directories, but in my opinion the full defrag is a waste of time if you do it more than once a month.

If you want to defragment with some kind of visual feedback, you can schedule the CLI version to run with a window [how?]

If you just want a fragmentation analysis, run defrag c: -a -v, which returns detailed statistics about the drive.  The results are not graphical, but in my opinion they are more useful than looking at a graphical map.  Again, a map can be misleading, since it may not give you a valid picture of what actual impact the fragmentation has on the drive.  The stats in the analysis includes things like the total available free space or the single largest contiguous free space on the drive, two things that are immensely important as far as fragmentation goes; the less free space you have, the more of a problem fragmentation is going to be and the harder it will be to defragment in the first place.

3. Use CONTIG to defragment individual files

If you want to defragment specific files, pick up the Contig utility from Sysinternals.  It's a little tool which defragments a single file through the Windows file-system APIs.  Contig is also used by the Power Defrgamenter utility, which can be used to defragment a single file, a whole directory, or a whole drive by using Contig.  It's most useful if you have a large file in several fragments that are not being defragged conventionally, but you don't want to go through the time and expense of running defrag with the -v switch to clean it up.  (Note that you'll probably need to run these programs as admin for them to work properly.)

4. Use a third-party, Vista-aware defragmentation app

I am putting this option last simply because I've grown less personally convinced that third-party defragmentation apps are crucial in a desktop environment.  The situation with servers is different, however; there's more than a few scenarios where a server can benefit from having some kind of fairly aggressive defragmentation.  (This is a topic that deserves its own discussion so I won't go into it here.)

Most of the commercial defrag programs have been fairly well-documented, like Diskeeper, but there are a few free ones that are worth mentioning -- JKDefrag, for instance, which does have a progress display and a great many other useful features besides.  One flaw: as of its current version it does not leave alone files tagged for prefetch, but the author is aware of this and may be including provisions for it in future releases.