Do ext4 filesystems need to be defragmented?

Question

Do ext4 filesystems need to be defragmented? If so, how do I defragment them?

If not, could you post a simple explanation of why they do not need to be defragmented?

Answer 1

Do ext4 filesystems need to be defragmented?

Yes (but very rarely).

If so, how do I defragment them?

Copy all the files off the partition, erase the files from the partition, then copy the files back onto the partition. The file system will intelligently allocate the files as you copy them back onto the disk.

If not, could you post a simple explanation of why they do not need to be defragmented?

ext4 acts in a more intelligent way than merely adding new files into the next available space. Instead of placing multiple files near each other on the hard disk, Linux file systems scatter different files all over the disk, leaving a large amount of free space between them. When a file is edited and needs to grow, there’s usually plenty of free space for the file to grow into. If fragmentation does occur, the file system will attempt to move the files around to reduce fragmentation in normal use, without the need for a defragmentation utility.

Thanks to a Comment by @Green Reaper my attention has been drawn to e4defrag.

Answer 2

I have been using ext4 partition for over a year. I often ran out of space, moved a lots of files in and out etc etc... these things are bad for fragmentation and yesterday I checked for fragmentation for the first time and only 2 files (not %, just two files) were fragmented.

On the Windows partition that was used for about a year longer with much more free space available, I have 95% fragmentation.

So no, you really don't need to defragment ext4 and if you want to be sure, leave the default free space for ext4 (default is 5%, can be changed by ex2tunefs -m X).

Answer 3

Here's an update from 2022

Fragmentation occurs due to multiple reasons:

• When an OS cannot manage file intelligently
• There's not enough continuous space to write the file
• When multiple applications write different files to the the disk at the same time

Reasons to defragment

Why was it needed to defragment files at all? On a HDD this is required to make access to files faster or move them to the area of the disk which provides higher speeds (the outer side of the platter). However there's another important thing many people are unaware of: when files are fragmented it becomes impossible to restore them when you intentionally or accidentally delete them since with so many fragments it's impossible to understand in which order they need to be reassembled. The normal sector size for ext4 is 4K so a 1GB file can theoretically have up to 262144 fragments. And before people say this can never happen I've had servers where our programmers wrote to two MySQL files simultaneously at approximately the same speed, so the resulting files were heavily fragmented and their read speed was well below 1MB/sec vs around 200MB/sec for "normal" files.

As for SSD there are almost no reasons to defragment but there are reasons not to do that since SSDs have a limited number of erase/write cycles. But that still requires an enormous amount of data to be written since modern SSDs feature quite large spare areas to alleviate the issue.

In terms of data recovery there's an issue. If you enabled the discard mount option deleted files are gone for good, so whether or not they were defragmented earlier it doesn't matter, you cannot physically recover them.

If you choose not to use this option and instead rely on a weekly job which does that (fstrim), then defragmenting makes sense.

Finding out the amount of fragmentation

I really doubt the previous comments were honest about the level of fragmentation that ext4 has. It can and it will heavily fragment files under certain circumstance, some applications like systemd-journald do that intentionally.

You can run e2fsck to check how heavily your partition is fragmented. On mounted partitions this will spew a ton of errors, please disregard them:

sudo e2fsck -n -v -f /dev/partition

Actually defragmenting

For the purpose of defragmenting ext4 I've written a script which needs to be run under root/sudo - you can get it here. It will defragment everything except the binaries of running applications - you may want to exit them beforehand. It needs a single argument to run - a path, e.g.

sudo defrag /

The script only defragments files on a given partition. If you have mounted partitions, specify them directly, i.e. sudo defrag /mnt/archive. It will probably won't defragment your swap file either, so running sudo swapoff -a prior to it is a necessity.

This will not defragment directories which can become fragmented as well. For that you'll need to run e2fsck on an unmounted partition:

sudo e2fsck -D -f -v -C 0 -n /dev/partition

Sometimes unmounting partitions becomes very difficult, please use any live Linux distro for that, e.g. System Rescue.

Free space defragmentation issue

The biggest issue with ext4 is that it's only capable of defragmenting individual files, thus you cannot defragment or make continuous free space (for this you'll want to use a different filesystem, e.g. XFS). It's perfectly possible to have a ton of free space yet being unable to defrag relatively small files.

E.g. my root FS for Fedora has 20GB of space, less than 7GB are occupied yet I cannot defrag a few binaries and libraries (chrome, libxul.so) which weigh in at less than 200MB. The reason? The average Linux distro contains tons of files (over 30K for my installation which is not even Gnome/KDE) and when you spread out these many files across the partition, there are not that many "holes" with a lot of space available.

Conclusion/TLDR

• If you have an HDD you probably want to defragment it
• If you have an SSD there's generally no need to defragment it unless you want to increase your chances of successful recovery of deleted files
• If you want to fully defragment your filesystem, you'll need a different filesystem, e.g. XFS.