by Han-Kwang Nienhuys, August 2008.
Is it necessary to defragment flash storage devices, such as a USB flash drives, SD cards, and newer SSD hard-disk replacements? Does it get faster if you do? There are arguments for and against defragmenting. On hard disk media, reading or writing fragmented files involves the overhead of the read heads having to jump from one place on the platter to the other, which can take as much as 8 milliseconds on a typical hard disk drive. However, hard disks are very fast when it comes to reading sequential data, which is on the order of 100 MB per second on a 3.5 inch consumer-grade hard disk. Obviously, allocating file data sequentially on a hard disk will save time on accessing the file.
Since a flash drive does not have moving parts, its access time is independent on where the data is stored, which is why supposedly flash drives don't need defragmenting. A quick Google search gives us plenty of pages where people suggest that there should be no difference in performance, while defragmentation only increases wear of the flash drive since flash memory can be rewritten only about 10 to 100 thousand times. Wear is usually not a big issue since most flash drives will use wear leveling to spread out the write operations, though. I found two pages where some empirical data is discussed, rather than theoretical considerations. One suggests that there is a small difference, which can be due to a flash device that contains both slow and fast flash memory, and due to I/O operations being needed for accessing non-contiguous data. [Yegulalp 2006] Another web page claims that there is a clear difference upon writing fragmented files. [Diskeeper 2007].
What these sources do not seem to realize is that reading flash memory can be done by random access, or on a sector (or page) level (512 to 2048 bytes at a time), but writing is done in much larger blocks of 16 to 256 kilobytes.[Wikipedia 2008]. Writing a single 512-byte sector of data can involve reading a full 128 kB block, erasing it, and then rewriting the original data with only 512 bytes changed.
I did a test on a Kingston 512 MB SD-card I bought in 2006 for my digital camera and tested it from a computer with SD card reader, running Linux. I created a highly fragmented file by reformatting the flash card (in FAT-32 format, 4 kB cluster size), and then sequentially writing thousands of files with sizes of 124 and 4 kB, interleaved, until the drive was full. Then I deleted all the small 4 kB files and replaced them by a single 15 MB file, that was spread out over about 4000 little fragments. I measured the time it took to write all this data, carefully flushing the disk cache every time with a sync command. Here are the results:
| Description | Data size | Time | Transfer speed |
|---|---|---|---|
| Initial write | 486 MB | 291 s | 1.67 MB/s |
| Fragmented write | 15.2 MB | 492.6 s | 0.031 MB/s |
That's pretty shocking: a factor 50 decrease in speed due to fragmentation! Apparently, a single write operation takes up to 120 milliseconds, even if it is only 4 kB of data.
After unmounting and remounting the flash drive (to get rid of data stored in the cache), I tested the read speeds:
| Description | Data size | Time | Transfer speed |
|---|---|---|---|
| Contiguous read | 12.1 MB | 1.7 s | 7.2 MB/s |
| Fragmented read | 15.2 MB | 4.6 s | 3.3 MB/s |
Clearly, there is some extra overhead for read operations on a fragmented file, but much, much less than for writing data.
Fragmentation can have a serious impact on flash drive performance, especially during write operations, which can take several milliseconds per data block. It might not be so important for a USB drive that is used to transfer a couple of files from one computer to the next, but if an SSD is used as a hard disk replacement, it can certainly be relevant. A standard hard disk defragmentation tool might not be the ideal tool to defragment a flash drive, since it is not aware of the allocation block size inside the flash drive. This may slow down the defragmentation procedure.
I will repeat the experiment later with other types of flash memory, and try to figure out the internal block size which is not specified anywhere.