Hello!

Now is time for something useful. A good backups solution that has proven to be effective during last XFS havoc - I was able to restore all damaged data. My weapon of choice is restic.

Some features🔗

• restic is a single statically-linked executable with no dependencies (yay! Portable!)
• restic does automatic data deduplication - repetetive data parts are stored just once
• restic has no concept of incremential backup. Each backup is full from user’s point of view.
• Each successive backup to the same target creates a snapshot of data. You may freely travel back in time, you may prepare retention scheme.
• restic backups are always encrypted (you can’t supply no/empty password)
• Since restic 0.14.0 there’s new repository format (v2), with compression enabled by default. Pre-14 backups are not automatically upgraded to the new version, you have to do it manually.
• You can restore data from any snapshot
• You can mount repository as a virtual read-only filesystem without restoring it and browse all files in all snapshots freely. Useful if you need one file and aren’t sure, which snapshot contains the version you need.
• restic can create backups to different targets:
  • local directory - on pendrive or removable HDD
  • SFTP server
  • REST service (they supply reference implementation)
  • S3 bucket (AWS, Wasabi or Alibaba, but also self-hosted Minio)
  • OpenStack Swift container
  • Azure Blob Storage
  • Google Cloud Storage
  • rclone bridge to over 70 other cloud storages

Basic usage🔗

Here comes rudimentary usage guide. For all examples I assume your backup media is mounted at /run/media/user/Backup (modern Linuxes mount removable media in /run/media/[LOGIN]/ with subdirectory mathich removable media label) and you want to backup your whole /home/user directory.

Giving location of repository🔗

You can point restic to backup repository in 2 ways:

• Directly, with --repo URI_OF_REPO
• With environment variable RESTIC_REPOSITORY

Giving password for repository🔗

You can provide a repository password in 4 ways:

• Do nothing and restic will ask for password as needed
• Give pasword in plaintext in RESTIC_PASSWORD environment variabe. The easiest way when you are not concerned about data theft.
• Store password in a file and provide path to it via RESTIC_PASSWORD_FILE variable or --password-file argument.
• Provide a password-asking program via RESTIC_PASSWORD_COMMAND variable or --password-command argument.

WARNING!, restic developers did their homework. If you forget your password, there’s no other way to unlock the repository. If you use a key file and you lose it - you’re royally fuc*ed.

You can share a single repository between multiple computers and/or OS-es, as long as their hostnames are all unique. They share a common password (or not - however all passwords are equivalent, grant all-or-nothing priviledge). The files are deduplicated also between hosts - so if you have the same big file in Linux and in Windows, a shared backup stores it only once.

Creating new, empty repo🔗

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

# Make repo
restic init

Updating repository with new data🔗

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

restic backup --verbose --compression max /home/user

• You can store few different directories in one snapshot (just list their paths separated with space)
• You can exclude directories containing a special CACHEDIR.TAG file containing a single line of Signature: 8a477f597d28d172789f06886806bc55 with --exclude-caches argument. Mor on the file itself is here. I find it way more covienient than manually creating exclude lists.

Listing snapshots from repository🔗

This will give you info on all snapshots - its identification (hex number), date created, host name that created it and list of paths stored in it

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

restic snapshots

Listing contents of a snapshot🔗

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

# Latest snapshot
restic ls latest

# Specific snapshot
restic ls SNAPSHOT_ID_IN_HEX

Restoring data🔗

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

# Latest snapshot
restic restore latest --target EMPTY_DIRECTORY_TO_RESTORE_TO

# Specific snapshot
restic restore SNAPSHOT_ID_IN_HEX --target EMPTY_DIRECTORY_TO_RESTORE_TO

• You can restore other host’s backup by giving its hostname with --host argument.
• You can restore just a specific path by giving it with --path argument.

Accessing data via mount🔗

You don’t select snapshot, all snapshots/hosts/tags are available at once as subdirectories. That’s the most useful functionality of restic. Note, this works even for network repositories!

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

restic mount EMPTY_DIRECTORY_TO_MOUNT_TO

You end restic mount mode witn ^C.

Cleaning old data🔗

Removing snapshots is a two-step process. First you select snapshot you don’t need (repeat as necessary):

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

restic forget SNAPSHOT_ID_IN_HEX

… then you purge the actual data that was referenced only by deleted snapshots (hence is no longer necessary)

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

restic prune

You can combine it into one step:

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

restic forget --prune SNAPSHOT_ID_IN_HEX 

You can also remove backups according to the time schedule, using --keep-last, --keep-hourly, --keep-daily, --keep-weekly, --keep-monthly, --keep-yearly or any combination thereof.

# Set location and password
export RESTIC_REPOSITORY=/run/media/user/Backup/MyBackup
export RESTIC_PASSWORD=MySecretPassw0rd

# Keep last week daily,
#      last month weekly
#      last year monthly
#      last 3 years yearly

restic forget --prune \
 --keep-daily 7 \
 --keep-weekly 4 \
 --keep-monthly 12 \
 --keep-yearly 3

By default each host is cleaned individually - your retention schedule is applied independently for each combination of hostname and backed-up path. You can change this behavior by --group-by argument. You can use any combination of host, tags and paths separated by comma. For example my setting is --group-by host,tags

A special form --group-by '' disables grouping altogether, which is not what you want.