Title: Making Qubes OS backups more efficient | |
Author: Solène | |
Date: 12 May 2023 | |
Tags: qubes qubesos backup | |
Description: In this article, you will learn how to send Qubes OS | |
backups to restic or borg backup software tools in order to make your | |
backups more efficient. | |
# Introduction | |
These days, I've been playing a lot with Qubes OS, it has an | |
interesting concept of deploying VMs (using Xen) in a well integrated | |
and transparent manner in order to hardly separate every tasks you | |
need. | |
By default, you get default environments such as Personal, Work and an | |
offline Vault, plus specials VMs to handle USB proxy, network and | |
firewall. What is cool here is that when you run a program from a VM, | |
only the window is displayed in your window manager (xfce), and not the | |
whole VM desktop. | |
The cool factor with this project is their take on the real world | |
privacy and security need, allowing users to run what they need to run | |
(proprietary software, random binaries), but still protect them. Its | |
goal is totally different from OpenBSD and Tails. Did I say you can | |
also route a VM network through Tor out of the box? =D | |
If you want to learn more, you can visit Qubes OS website (or ask if | |
you want me to write about it): | |
Qubes OS official website | |
New user guide: How to organize your qubes (nice reading to understand Qubes OS) | |
# Backups | |
If you know me, you should know I'm really serious about backups. This | |
is incredibly important to have backups. | |
Qubes OS has a backup tool that can be used out of the box, it just | |
dump the VMs storage into an encrypted file, it's easy but not | |
efficient or practical enough for me. | |
If you want to learn more about the format used by Qubes OS (and how to | |
open them outside of Qubes OS), they wrote some documentation: | |
Qubes OS: backup emergency restore | |
Now, let's see how to store the backups in Restic or Borg in order to | |
have proper backups. | |
/!\ While both software support deduplication, this doesn't work well | |
in this case because the stored data are compressed + encrypted | |
already, which has a very high entropy (it's hard to find duplicated | |
patterns). | |
# Backup tool | |
Qubes OS backup tool offers compression and encryption out of the box, | |
but when it comes to the storage location, we can actually use a | |
command to send the backups to the command's stdin, and guess what, | |
both restic and borg support receiving data on their standard input! | |
I'll demonstrate how to proceed both with restic and borg with a simple | |
example, I recommend to build your own solution on top of it the way | |
you need. | |
Screenshot of Qubes backup tool | |
# Create a backup VM | |
As we are running Qubes OS, I prefer to create a dedicated backup VM | |
using the Fedora template, it will contain the passphrase to the | |
repository and an SSH key for remote backup. | |
You need to install restic/borg in the template to make it available in | |
that VM. | |
If you don't know how to install software in a template, it's well | |
documented: | |
Qubes OS: how to install software | |
Generate an SSH key if you want to store your data on a remote server | |
using SSH, and deploy it on the remote server. | |
# Write a backup script | |
In order to simplify the backup command configuration in the backup | |
tool (it's a single input line), but don't sacrifice on features like | |
pruning, we will write a script on the backup VM doing everything we | |
need. | |
While I'm using a remote repository in the example, nothing prevents | |
you from using a local/external drive for your backups! | |
The script usage will be simple enough for most tasks: | |
* `./script init` to create the repository | |
* `./script backup` to create the backup | |
* `./script list` to display snapshots | |
* `./script restore $snapshotID` to restore a backup, the output file | |
will always be named `stdin` | |
## Restic | |
Write a script in `/home/user/restic.sh` in the backup VM, it will | |
allow simple customization of the backup process. | |
```shell | |
#!/bin/sh | |
export RESTIC_PASSWORD=mysecretpass | |
# double // is important to make the path absolute | |
export RESTIC_REPOSITORY=sftp://[email protected]://var/backups/restic_qubes | |
KEEP_HOURLY=1 | |
KEEP_DAYS=5 | |
KEEP_WEEKS=1 | |
KEEP_MONTHS=1 | |
KEEP_YEARS=0 | |
case "$1" in | |
init) | |
restic init | |
;; | |
list) | |
restic snapshots | |
;; | |
restore) | |
restic restore --target . $2 | |
;; | |
backup) | |
cat | restic backup --stdin | |
restic forget \ | |
--keep-hourly $KEEP_HOURLY \ | |
--keep-daily $KEEP_DAYS \ | |
--keep-weekly $KEEP_WEEKS \ | |
--keep-monthly $KEEP_MONTHS \ | |
--keep-yearly $KEEP_YEARS \ | |
--prune | |
;; | |
esac | |
``` | |
Obviously, you have to change the password, you can even store it in | |
another file and use the according restic option to load the passphrase | |
from a file (or from a command). Although, Qubes OS backup tool | |
enforces you to encrypt the backup (which will be store in restic), so | |
encrypting the restic repository won't add any more security, but it | |
can add privacy by hiding what's in the repo. | |
/!\ You need to run the script with the parameter "init" the first | |
time, in order to create the repository: | |
```shell | |
$ chmod +x restic.sh | |
$ ./restic.sh init | |
``` | |
## Borg | |
Write a script in `/home/user/borg.sh` in the backup VM, it will allow | |
simple customisation of the backup process. | |
```shell | |
#!/bin/sh | |
export BORG_PASSPHRASE=mysecretpass | |
export BORG_REPO=ssh://[email protected]/var/solene/borg_qubes | |
KEEP_HOURLY=1 | |
KEEP_DAYS=5 | |
KEEP_WEEKS=1 | |
KEEP_MONTHS=1 | |
KEEP_YEARS=0 | |
case "$1" in | |
init) | |
borg init --encryption=repokey | |
;; | |
list) | |
borg list | |
;; | |
restore) | |
borg extract ::$2 | |
;; | |
backup) | |
cat | borg create ::{now} - | |
borg prune \ | |
--keep-hourly $KEEP_HOURLY \ | |
--keep-daily $KEEP_DAYS \ | |
--keep-weekly $KEEP_WEEKS \ | |
--keep-monthly $KEEP_MONTHS \ | |
--keep-yearly $KEEP_YEARS | |
;; | |
esac | |
``` | |
Same explanation as with restic, you can save the password elsewhere or | |
get it from a command, but Qubes backup already encrypt the data, so | |
the repo encryption will mostly only add privacy. | |
/!\ You need to run the script with the parameter "init" the first | |
time, in order to create the repository: | |
```shell | |
$ chmod +x borg.sh | |
$ ./borg.sh init | |
``` | |
## Configure Qubes backup | |
Now, configure the Qubes backup tool: | |
* Choose the VMs to backup | |
* Check "Compress backups", because it's done before encryption it | |
yields a better efficiency than compression done by restic on the | |
encrypted data | |
* Click Next | |
* Choose the backup VM in the "Target qube" list | |
* In the field "backup directory or command" type `/home/user/restic.sh | |
backup` or `/home/user/borg.sh backup` depending on your choice | |
* Pick a passphrase | |
* Run the backup | |
# Restoring a backup | |
While it's nice to have backups, it's important to know how to use | |
them. The setup doesn't add much complexity, and the helper script | |
will ease your life. | |
On the backup VM, run `./borg.sh list` (or the restic version) to | |
display available snapshots in the repository, then use `./borg.sh | |
restore $snap` with the second parameter being a snapshot identifier | |
listed in the earlier command. | |
You will obtain a file named `stdin`, this is the file to use in Qubes | |
OS restore tool. | |
# Warning | |
If you don't always backup all the VMs, if you keep the retention | |
policy like in the example above, you may lose data. | |
For example, if you have a KEEP_HOURLY=1, create a backup of all your | |
VMs, and just after, you specifically want to backup a single VM, you | |
will lose the previous full backup due to the retention policy. | |
In some cases, it may be better to not have any retention policy, or | |
simply time based (keep snapshots which date < n days). | |
# Conclusion | |
Using this configuration, you get all the features of a industry | |
standard backup solution such as integrity check, retention policy or | |
remote encrypted storage. | |
# Troubleshoot | |
In case of an issue with the backup command, Qubes backup will display | |
a popup message with the command output, this helps a lot debugging | |
problems. | |
An easy way to check if the script works by hand is to run it from the | |
backup VM: | |
```shell | |
echo test | ./restic.sh backup | |
``` | |
This will create a new backup with the data "test" (and prune older | |
backups, so take care!), if it doesn't work this is a simple way to | |
trigger a new backup to solve your issue. |