Encrypted Backups with restic

Tested on: Ubuntu 24.04 LTS and AlmaLinux 9.4, restic 0.16.x. The patterns translate to restic 0.15+ unchanged; older versions have a few CLI differences flagged inline.

Why this matters

The most common “we had backups, but…” incidents reduce to one of four failures:

1. The backup was on the same disk as the data. When the disk failed, both went together.
2. The backup tool never encrypted anything. A misconfigured S3 bucket leaked the entire database — including the personal data the business had promised to protect.
3. The encryption passphrase died with the original server. The off-site backups still exist; nobody can read them.
4. The backup ran nightly and the most recent twelve failed silently. No alert, no log review, no drill.

restic addresses every one of these as a default. It encrypts in the client, deduplicates, supports a wide range of off-site targets, exposes a clean check command for integrity, and reports non-zero exit codes on failure that integrate cleanly with cron and CI. It is not the only acceptable choice — borgbackup is the close runner-up, and for some local-first workloads it is the better fit — but for the EEA-region, S3-compatible, off-site target this site recommends, restic is the path of least resistance.

This guide is the deeper dive referenced from postgres-backups and privacy-by-design-server-build. Where those guides treat restic as “step 4: off-site”, this one covers the operational details that turn a daily push into a defensible recovery story.

1. Install restic

1# Debian / Ubuntu
2sudo apt update && sudo apt install -y restic
3
4# RHEL / AlmaLinux (via EPEL)
5sudo dnf install -y epel-release && sudo dnf install -y restic
6
7restic version    # confirm 0.15+ ideally 0.16+

The packaged version is often a few releases behind. If you need a recent feature (e.g. compression, added in 0.14), install the upstream binary instead:

1sudo restic self-update

2. Decide where the repository lives

A restic repository is a directory of encrypted, content-addressed blobs. Pick the location before you initialise:

For the rest of this guide we use an S3-compatible endpoint as the canonical example, named in the privacy notice’s processor table. Any EEA-region S3-compatible service works — the commands do not change.

3. Passphrase strategy — the load-bearing decision

This is the single most important choice in the whole setup. A restic repository is encrypted symmetrically; lose the passphrase, lose the backups. There is no recovery, by design. Three rules:

1. Long and random. 30+ characters, generated by a password manager. A memorable phrase is a security regression here — the passphrase travels in a file, not in your head.
2. Stored outside the original server. Password manager that survives the loss of the server. Treat it like the root TOTP seed for a domain registrar.
3. Known to at least two people. A single-passphrase-holder is one bus-incident away from un-recoverable backups.

Concretely:

1# Generate a passphrase in your password manager (or with openssl).
2openssl rand -base64 33 | tr -d '/+=' | head -c 40
3# → store in 1Password / Bitwarden / Vault under
4#   "stackharden / production backups / restic"
5
6# On the server, write it to a file the backup user can read.
7sudo mkdir -p /etc/restic
8sudo install -o root -g root -m 600 /dev/stdin /etc/restic/passphrase.txt <<< 'PASTE_HERE'

The local file at /etc/restic/passphrase.txt is the operational copy. The password manager is the authoritative copy. If the file is deleted, the password manager restores it. If the server dies, the password manager is what you reach for.

4. Initialise the repository

1export RESTIC_REPOSITORY=s3:s3.eu-central-1.amazonaws.com/stackharden-backups
2export RESTIC_PASSWORD_FILE=/etc/restic/passphrase.txt
3export AWS_ACCESS_KEY_ID=...
4export AWS_SECRET_ACCESS_KEY=...
5
6restic init

Notes:

• The S3 bucket must already exist. restic does not create it.
• Use a dedicated S3 IAM user with PutObject, GetObject, DeleteObject, ListBucket on this bucket only. Not the account-wide credentials.
• Server-side encryption (SSE-S3 / SSE-KMS) is optional and additive. The data is already client-side encrypted; SSE is a defence in depth against a misconfigured bucket policy, not a substitute for restic’s own encryption.

5. The daily backup script

/usr/local/bin/restic-backup:

 1#!/usr/bin/env bash
 2set -euo pipefail
 3
 4# Configuration lives in /etc/restic/env — sourced from systemd or cron.
 5: "${RESTIC_REPOSITORY:?}"
 6: "${RESTIC_PASSWORD_FILE:?}"
 7
 8TAG="$(hostname)"
 9LOCK_HOLD=600        # seconds — abort a stale lock from a failed run
10
11# 1. Drop a stale lock if one exists.
12restic unlock --no-cache || true
13
14# 2. Run the backup. --tag groups snapshots per host so retention
15#    can be per-host even in a shared repository.
16restic backup \
17    --tag "$TAG" \
18    --exclude-file=/etc/restic/excludes \
19    --one-file-system \
20    /etc /home /srv /var/log /var/backups
21
22# 3. Apply retention. --prune removes blobs unreferenced by any
23#    retained snapshot.
24restic forget \
25    --tag "$TAG" \
26    --keep-daily 7 \
27    --keep-weekly 4 \
28    --keep-monthly 12 \
29    --prune
30
31# 4. Verify the repository structure (cheap — metadata only).
32restic check --no-cache

/etc/restic/excludes excludes paths that are noise or dangerous:

# Caches and ephemeral data
**/node_modules
**/.cache
**/__pycache__
/var/log/journal/**
/var/cache/**
/tmp/**

# Other people's backups (avoid double-backing)
/var/lib/postgresql/16/backups/**

# Anything containing live secrets you have committed to never copying

Schedule via systemd timer (preferred — it logs to the journal and supports OnFailure=) or cron. The script’s set -euo pipefail plus restic’s own exit codes mean a failure is loud, not silent.

6. Retention — and matching the policy you publish

The numbers above (7 daily / 4 weekly / 12 monthly) are a defensible starting point. They must reconcile with the user-data retention period in your privacy notice — see privacy-by-design-server-build for the reasoning. Common adjustments:

• Cut the monthly retention if your privacy policy promises “we delete user data within 30 days of account closure” and you do not have a legal basis for keeping it longer.
• Extend the weekly retention for environments under legal hold.
• Add --keep-yearly only if you have a specific compliance obligation that requires it. Yearly retention compounds — five yearly snapshots of a 100 GB system is more storage than people expect.

7. Verification — restic check is not enough

There are three levels of verification, each with a different cost:

Recommended cadence:

• Daily: metadata check (already in the backup script).
• Weekly: 5% sample check from cron.
• Monthly: full read-data verification. Schedule for off-peak.

A read-data check is the only way to detect bit-rot at the storage provider — file integrity at the source plus the provider’s checksums do not, in practice, catch every case.

8. Restoration drills

A backup is not a backup until it has been restored. The drill, at minimum quarterly:

 1# 1. Pick a snapshot.
 2restic snapshots --tag "$(hostname)" | tail
 3
 4# 2. Restore to a temp directory.
 5sudo mkdir -p /var/restore-test
 6restic restore <snapshot-id> --target /var/restore-test
 7
 8# 3. Sanity-check key files.
 9ls -la /var/restore-test/etc/passwd
10# ... and whatever else matters for your stack
11
12# 4. Time it. RTO is the metric, not RPO.

Record the date, the snapshot ID, the time-to-restore, and any anomalies. A spreadsheet line per drill is the level of documentation an auditor expects to see.

Once a year, do the drill on a different host, restoring to a fresh VPS — that is the only way to verify the passphrase survives the loss of the original server.

9. Monitoring backup health

A silent backup failure is the default failure mode. Wire alerting on at least:

• Non-zero exit from the backup script. systemd OnFailure= to an email or webhook unit; cron piping to logger -p user.err and alerting on the syslog channel.
• restic check non-zero. Same as above; the metadata check inside the backup script handles this.
• Repository quiet for >36 hours. The most insidious failure is a cron job that stops firing. A scheduled job elsewhere — your monitoring host, a hosted cron — that queries restic snapshots and alerts on the most recent timestamp being older than the expected interval.

restic stats --mode raw-data produces machine-readable output suitable for Prometheus scraping if you have a Prometheus textfile collector available.

10. Restoring a single file or partial directory

The common case is not “everything is gone” — it is “I just deleted the wrong file”:

 1# List versions of a file across snapshots
 2restic find /etc/nginx/nginx.conf
 3
 4# Restore a single file from a specific snapshot
 5restic restore <snapshot-id> --target /tmp/restore \
 6    --include /etc/nginx/nginx.conf
 7
 8# Or mount the snapshot read-only for browsing
 9mkdir /mnt/restic
10restic mount /mnt/restic
11# Browse via /mnt/restic/snapshots/<id>/...
12# Ctrl-C in the mount terminal to unmount.

restic mount requires FUSE; on hardened hosts that is sometimes not available, in which case restore --include is the path.

Gotchas

Passphrase rotation is not a normal operation

restic supports adding additional passwords to a repository (restic key add) and removing existing ones (restic key remove). This is useful when staff turnover means a previous passphrase has walked out of the door. It is not a routine rotation activity — each key add creates a new master-key wrapper but does not re-encrypt data. The cost of full passphrase rotation is a full re-backup.

restic prune and contention

restic prune rewrites data and cannot run concurrently with backup. The retention pattern in the daily script (forget --prune) acquires the repository lock during pruning, which on large repositories can block the next scheduled backup. For repositories above ~100 GB, split: daily forget (cheap), weekly prune (heavier, off-peak).

Skipped --one-file-system with bind mounts

--one-file-system is a sensible default — it stops restic descending into network mounts, FUSE mounts, and /proc / /sys. But it also stops descent into bind mounts that legitimately need backing up. Audit your /etc/fstab for bind mounts before relying on the flag.

Backup target credentials with too much scope

The dedicated S3 IAM user should have privileges scoped to the single bucket. A common mistake is reusing the application’s S3 credentials — which often have wider scope — for backup. A compromise of the production app then leaks the backup target’s credentials too.

Repository in the same blast radius as the source

A “backup” S3 bucket in the same cloud account, same region, same billing entity as the production VPS is one stolen credential away from being deleted alongside the production data. Cross-account or cross-provider backup is a meaningful upgrade — at minimum, enable versioning + MFA-delete on the backup bucket so a credential compromise cannot wipe history.

What this guide deliberately does not cover

• borgbackup as an alternative — equally credible, different ergonomics; a comparison guide is on the backlog.
• Database-specific backup orchestration — see postgres-backups for pg_dump / WAL archiving feeding restic.
• Bare-metal restore — restic restores files; full system recovery from bare metal also needs a known-good base image plus a documented order of operations. Out of scope here.
• Restic Server (the optional REST endpoint that fronts a filesystem repository) — useful in private networks; a separate guide.