Search
Write a publication
Pull to refresh

My way of a full system backup without external software: incremental rsync plus btrfs with zstd compression

Level of difficultyMedium
Reading time3 min
Views150

The repo of this script is https://gitlab.com/vitaly‑zdanevich/full‑backup/‑/blob/master/full‑backup.sh

Incremental with hard links means that if a file is not changed, on the next backup it will link to the same underlying data, like deduplication. Hard links — its usual files.

Also, this script ignores .gitignore of every folder.

Run this script from another system.

# Start this script from the git folder of this script
# This script accepts the path to the partition to backup (mounted)
# Use this script NOT from the system you want to backup - boot from another system

if [ -z "$1" ]; then
	echo "Missing path of the source (what to backup)"
	exit 1
fi

BEFORE=$(df -h)

STARTED=$(date)

DATE=`date "+%Y-%m-%d"`

p=$(pwd)

cd $1
# Because rsync --exclude-from accepts only relative paths, not absolute

rsync --archive --acls --xattrs --delete --progress --verbose --exclude-from=$p/exclude.txt --filter=':- .gitignore' --link-dest=$p/last --mkpath $1 $p/$DATE

ln --symbolic --force --no-dereference $DATE last

echo "Started at:   " $STARTED
echo "Current time: " $(date)

echo "Before:

$BEFORE

Now:
"

df -h

cd -

# How to restore:
#
# rsync --archive --acls --xattrs --progress --verbose <from> <to>
# /etc/fstab: alter UUID
# grub install (if new SSD) https://wiki.gentoo.org/wiki/Grub:
#     grub-install
# Note that if SSD is more than 2 GB - you cannot use MBR, only GPT with EFI partition (ThinkPad T430 supports UEFI boot)
# emerge-webrsync
# emaint sync guru


# Documentation:
# https://jumpcloud.com/blog/how-to-backup-linux-system-rsync
# https://wiki.archlinux.org/title/rsync

exclude.txt:

dev/*
proc/*
sys/*
run/*
# If run on the system that you want to backup

var/db/repos/gentoo
var/db/repos/guru
var/cache/distfiles/

tmp/*
var/tmp

lost+found
mnt/*

home/vitaly/.npm/
home/vitaly/.cache/

home/vitaly/.zoom/

home/vitaly/.mozilla/firefox/*/storage
home/vitaly/.config/google-chrome/
home/vitaly/.config/chromium/
home/vitaly/.config/Microsoft/Microsoft Teams/
home/vitaly/.config/teams-for-linux/
home/vitaly/.config/Slack/
home/vitaly/.thumbnails/

home/vitaly/.cache/
home/vitaly/.local/share/TelegramDesktop/tdata/user_data/cache/
home/vitaly/.local/share/TelegramDesktop/tdata/user_data/media_cache/
home/vitaly/.local/share/Steam/
home/vitaly/.googleearth/Cache/
home/vitaly/.local/share/OMaps/
home/vitaly/.config/Audius/Cache/
home/vitaly/.config/YandexMusic/Cache/
home/vitaly/.config/vesktop/sessionData/Cache/
home/vitaly/.config/bilibili/Cache/
home/vitaly/.local/share/Trash/

home/vitaly/go/pkg/mod/cache/
home/vitaly/.cargo/registry/

home/vitaly/record/out/

home/vitaly/Desktop/unreal-5-4-4/

My HDD for backups is on btrfs for ZSTD compression, how to prepare the disk:

sudo apt install btrfs-progs -y
# https://details.nl/install-and-use-btrfs-on-ubuntu-20-04-lts

sudo mkfs.btrfs /dev/sdX
# Partitions are not needed

sudo mount -o compress-force=zstd:15 /dev/sdX /mnt
# Max compression level is 15
# Default compression you can add to fstab

After backup — you can see the compression ratio:

sudo apt install btrfs-compsize
sudo compsize /mnt

Output example:

   Processed 54036 files, 42027 regular extents (42028 refs), 27150 inline.
   Type       Perc     Disk Usage   Uncompressed Referenced
   TOTAL       73%      211G         289G         289G
   zstd        28%       42M         148M         148M
   none       100%      174G         174G         174G

The main column here is Perc — lower is better compression.

Besides this backup solution, I store every config in a separate git repository, not one repo for all dot files.

Sometimes I use Clonezilla — with xz compression of the full drive, for Windows and Linux.

This article is based on my Russian article https://habr.com/en/articles/929182, where people also recommended their backup solutions:

https://github.com/jimsalterjrs/sanoid/wiki/Syncoid

https://github.com/jimsalterjrs/sanoid

https://github.com/bit‑team/backintime

https://github.com/ei‑grad/trinkup

https://restic.net

https://www.urbackup.org

https://www.borgbackup.org

https://kopia.io

https://duplicati.com

http://dar.linux.free.fr

BTRFS snapshots

Tags:
Hubs:
+3
Comments0

Articles