Migrating Docker Data to HDD: Save Your SSD from Unnecessary Wear

With the rising cost of computer components, I felt compelled to write this article. It seems there's a demand from above.

We programmers are a generally lazy bunch. Most of the time, we use things straight out of the box, as they are. The situation described below is no exception.

I don't work with Docker daily, but I've known it for a long time. Over roughly five years, I've faced the issue of disappearing disk space several times. And what was the solution? Correct: docker system prune or similar commands. Space cleared—problem solved.

But honestly, this isn't good for SSD and similar drives. To extend a drive's longevity, it's necessary to keep it at most 70% full (I don't remember the exact figure), and ideally around 50%. Essentially, half the disk should always be empty. If you "clog up" the disk with cache and volumes, which are like zombie processes in Linux (i.e., not attached to anything but still occupying space), it won't last long. And buying a drive that costs three times more isn't very appealing. Let's be direct: the "Docker workflow" is mostly needed to temporarily show us something. Later, containers get rebuilt, and everything gets rewritten. Speed isn't critical here either because there's a cache that speeds up builds. This makes it the perfect candidate for moving to an HDD.

In short, like most programmers, for years I've been lazy and just used docker system prune. But recently, on a whim, I decided to check how Docker was doing after just a week or two on the disk. docker system df showed me that a total of 110GB had already been consumed!

That was the last straw. Moreover, right next to me was an idle HDD. The decision was made to put it to work.

As always, the lazier you are, the longer things take. So here, once again, I was lazy and decided to use an existing disk with NTFS formatting. This led to wasted time and some inconveniences with permissions and other issues.

After all this, once the realization dawned that it's better not to be lazy in this case, it was decided to allocate a special area from the HDD just for Docker. I allocated 300GB, which should be sufficient in my case, but this time I formatted the disk partition as ext4 since I use Arch Linux (Manjaro), which suits almost everyone except Windows developers.

And what a surprise it was when the replacement wasn't that long after all. As always, the thought everyone knows: why didn't I do this earlier?!

Below are a few lines that protect your SSD from unnecessary load:

• docker info | grep "Docker Root Dir" - Verify everything is at the default location.

• lsblk -f - Find the disk UUID; this disk will be used purely for storing Docker information (volumes, images, etc.), and nothing else should be on it.

• sudo mkfs.ext4 -L docker-disk /dev/sda1 - Create an ext4 filesystem partition if you haven't already.

• sudo blkid /dev/sda1 - Verify that it worked.

• sudo mkdir -p /mnt/docker-data - Now create a mount point.

• echo 'UUID=3b222345-XXXX-XXXX-2ca9ce3040fd /mnt/docker-data ext4 defaults,noatime 0 2' | sudo tee -a /etc/fstab - Add the disk with your UUID.

• sudo mount -a

• sudo systemctl stop docker.socket && sudo systemctl stop docker.service && sudo systemctl stop docker && sudo rsync -aqxP /var/lib/docker/ /mnt/docker-data/ - Migrate data from the existing Docker installation.

• sudo chown -R root:root /mnt/docker-data && sudo chmod -R 755 /mnt/docker-data - Set the correct permissions.

• sudo mkdir -p /etc/docker - Create a directory for settings; I didn't have one.

• Create and fill the configuration file:

  • sudo touch /etc/docker/daemon.json

  • sudo nano /etc/docker/daemon.json (or use your preferred editor like mousepad):

      {
          "data-root": "/mnt/docker-data",
          "storage-driver": "overlay2"
      }
    

• sudo systemctl start docker

That's essentially it. Everything worked for me.

Important Note: If the disk is disconnected, the system will likely hang during boot with these settings. To allow the system to boot without the disk, the minimum step is to remove the added disk entry from the /etc/fstab file, so the system won't wait for it at startup. And yes, adding the nofail option to fstab didn't help in my case.