Space effective backups of Raspberry Pi SD-cards

RaspberryPi mini computers are great for all kinds of home automation, media centers, home servers. Yet it should be known that the draw back for permanent operation can be the SD-card. Especially when applications on the RasperrbyPi cause excessive read/writes or if it is used outsides where temperatures can reach the SD-Card operating limits, IO errors should be expected that can just f* up the operating system.

The easiest way to be prepared is to have a full SD-card image of the RaspberryPi at hand! The easiest way is to clone the SD card for example using Win32 Disk Imager.

The resulting image is as large as the SD-card. As the card is usually not 100% in use, the image size should considerably decrease by compression. Yet it is not uncommon that a 16 GB image with 3 GB data cannot be compressed to less than 10 GB! The reason is that deleting files only removes the file handles and leaves the data unreferenced. So: the longer the SD-card was in use (=the more files were written / deleted on it), the worse the compression factor.

How to make a well compressable SD-card image

The trick simply to wipe the free space! An easy way is to fill the free space with zeros. When the image is compressed later, all zeros can be compressed to almost nothing.

  1. Open a “monitoring shell” and type: while (true) do df -h /; sleep 1; done
    Use this shell to monitor the free space of the SD card while it is filled.
  2. Open a second shell: dd if=/dev/zero of=/tmp/zero1.tmp bs=1048576 count=4000. This creates a 4 GB file, filled with just zeros. Adjust count to a size that fills (almost) all the available free space on the SD-card (or create multiple files).
  3. Cancel the dd command (ctrl-c) before it fills ALL the free space (keep an eye on the “monitoring shell”).
  4. Delete the zero-files and shut down the Raspi. The free space is now wiped!
  5. Read (Win32 Disk Imager) and compress the image.

The compressed image can be up to 10x smaller than without wiping the free space!

Leave a Reply

Your email address will not be published. Required fields are marked *