Title: Declaratively manage your Qubes OS

	Title: Declaratively manage your Qubes OS
	Author: Solène
	Date: 02 June 2023
	Tags: qubesos salt qubes
	Description: In this article, you will learn how to use Qubes OS
	internal salt stack configuration management to manage it
	programmatically

	# Introduction

	As a recent Qubes OS user, but also a NixOS user, I want to be able to
	reproduce my system configuration instead of fiddling with files
	everywhere by hand and being clueless about what I changed since the
	installation time.

	Fortunately, Qubes OS is managed internally with Salt Stack (it's
	similar to Ansible if you didn't know about Salt), so we can leverage
	salt to modify dom0 or Qubes templates/VMs.

	Qubes OS official project website
	Salt Stack project website
	Qubes OS documentation: Salt

	# Simple setup

	In this example, I'll show how to write a simple Salt state files,
	allowing you to create/modify system files, install packages, add
	repositories etc...

	Everything will happen in dom0, you may want to install your favorite
	text editor in it. Note that I'm still trying to figure a nice way to
	have a git repository to handle this configuration, and being able to
	synchronize it somewhere, but I still can't find a solution I like.

	The dom0 salt configuration can be found in `/srv/salt/`, this is where
	we will write:

	* a .top file that is used to associate state files to apply to which
	hosts
	* a state file that contain actual instructions to run

	Quick extra explanation: there is a directory `/srv/pillar/`, where you
	store things named "pillars", see them as metadata you can associate to
	remote hosts (AppVM / Templates in the Qubes OS case). We won't use
	pillars in this guide, but if you want to write more advanced
	configurations, you will surely need them.

	# dom0 management

	Let's use dom0 to manage itself 🙃.

	Create a text file `/srv/salt/custom.top` with the content (YAML
	format):

	```yaml
	base:
	'dom0':
	- dom0
	```

	This tells that hosts matching `dom0` (2nd line) will use the state
	named `dom0`.

	We need to enable that .top file so it will be included when salt is
	applying the configuration.

	```command
	qubesctl top.enable custom
	```

	Now, create the file `/srv/salt/dom0.sls` with the content (YAML
	format):

	```yaml
	my packages:
	pkg.installed:
	- pkgs:
	- kakoune
	- git
	```

	This uses the salt module named `pkg`, and pass it options in order to
	install the packages "git" and "kakoune".

	Salt Stack documentation about the pkg module

	On my computer, I added the following piece of configuration to
	`/srv/salt/dom0.sls` to automatically assign the USB mouse to dom0
	instead of being asked every time, this implements the instructions
	explained in the documentation link below:

	Qubes OS documentation: USB mice

	```yaml
	/etc/qubes-rpc/policy/qubes.InputMouse:
	file.line:
	- mode: ensure
	- content: "sys-usb dom0 allow"
	- before: "^sys-usb dom0 ask"
	```

	Salt Stack documentation: file line

	This snippet makes sure that the line `sys-usb dom0 allow` in the file
	`/etc/qubes-rpc/policy/qubes.InputMouse` is present above the line
	matching `^sys-usb dom0 ask`. This is a more reproducible way of
	adding lines to configuration file than editing by hand.

	Now, we need to apply the changes by running salt on dom0:

	```command
	qubesctl --target dom0 state.apply
	```

	You will obtain a list of operations done by salt, with a diff for each
	task, it will be easy to know if something changed.

	Note: state.apply used to be named state.highstate (for people who used
	salt a while ago, don't be confused, it's the same thing).

	# Template management

	Using the same method as above, we will add a match for the fedora
	templates in the custom top file:

	In `/srv/salt/custom.top` add:

	```yaml
	'fedora-*':
	- globbing: true
	- fedora
	```

	This example is slightly different than the one for dom0 where we
	matched the host named "dom0". As I want my salt files to require the
	least maintenance possible, I won't write the template name verbatim,
	but I'd rather use a globbing (this is the name for simpler wildcard
	like `foo*`) matching everything starting by `fedora-`, I currently
	have fedora-37 and fedora-38 on my computer, so they are both matching.

	Create `/srv/salt/fedora.sls`:

	```yaml
	custom packages:
	pkg.installed:
	- pkgs:
	- borgbackup
	- dino
	- evolution
	- fossil
	- git
	- pavucontrol
	- rsync
	- sbcl
	- tig
	```

	In order to apply, we can type `qubesctl --all state.apply`, this will
	work but it's slow as salt will look for changes in each VM / template
	(but we only added changes for fedora templates here, so nothing would
	change except for the fedora templates).

	For a faster feedback loop, we can specify one or multiple targets, for
	me it would be `qubesctl --targets fedora-37,fedora-38 state.apply`,
	but it's really a matter of me being impatient.

	# Auto configure Split SSH

	An interesting setup with Qubes OS is to have your SSH key in a
	separate VM, and use Qubes OS internal RPC to use the SSH from another
	VM, with a manual confirmation on each use. However, this setup
	requires modifying files at multiple places, let's see how to manage
	everything with salt.

	Qubes OS community documentation: Split SSH

	Reusing the file `/srv/salt/custom.top` created earlier, we add
	`split_ssh_client.sls` for some AppVMs that will use the split SSH
	setup. Note that you should not deploy this state to your Vault, it
	would self reference for SSH and would prevent the agent to start (been
	there :P):

	```
	base:
	'dom0':
	- dom0
	'fedora-*':
	- globbing: true
	- fedora
	'MyDevAppVm or MyWebBrowserAppVM':
	- split_ssh_client
	```

	Create `/srv/salt/split_ssh_client.sls`: this will add two files to
	load the environment variables from `/rw/config/rc.local` and
	`~/.bashrc`. It's actually easier to separate the bash snippets in
	separate files and use `source`, rather than using salt to insert the
	snippets directly in place where needed.

	```
	/rw/config/bashrc_ssh_agent:
	file.managed:
	- user: root
	- group: wheel
	- mode: 444
	- contents: \|
	SSH_VAULT_VM="vault"
	if [ "$SSH_VAULT_VM" != "" ]; then
	export SSH_AUTH_SOCK="/home/user/.SSH_AGENT_$SSH_VAULT_VM"
	fi

	/rw/config/rclocal_ssh_agent:
	file.managed:
	- user: root
	- group: wheel
	- mode: 444
	- contents: \|
	SSH_VAULT_VM="vault"
	if [ "$SSH_VAULT_VM" != "" ]; then
	export SSH_SOCK="/home/user/.SSH_AGENT_$SSH_VAULT_VM"
	rm -f "$SSH_SOCK"
	sudo -u user /bin/sh -c "umask 177 && exec socat 'UNIX-LISTEN:$SSH_SO…
	fi

	/rw/config/rc.local:
	file.append:
	- text: source /rw/config/rclocal_ssh_agent

	/rw/home/user/.bashrc:
	file.append:
	- text: source /rw/config/bashrc_ssh_agent
	```

	Edit `/srv/salt/dom0.sls` to add the SshAgent RPC policy:

	```
	/etc/qubes-rpc/policy/qubes.SshAgent:
	file.managed:
	- user: root
	- group: wheel
	- mode: 444
	- contents: \|
	MyClientSSH vault ask,default_target=vault
	```

	Now, run `qubesctl --all state.apply` to configure all your VMs, which
	are the template, dom0 and the matching AppVMs. If everything went
	well, you shouldn't have errors when running the command.

	# Use a dedicated AppVM for web browsing

	Another real world example, using Salt to configure your AppVMs to open
	links in a dedicated AppVM (named WWW for me):

	Qubes OS Community Documentation: Opening URLs in VMs

	In your custom top file `/srv/salt/custom.top`, you need something
	similar to this (please adapt if you already have top files or state
	files):

	```
	'dom0':
	- dom0
	'fedora-*':
	- globbing: true
	- fedora
	'vault or qubes-communication or qubes-devel':
	- default_www
	```

	Add the following text to `/srv/salt/dom0.sls`, this is used to
	configure the RPC:

	```yaml
	/etc/qubes-rpc/policy/qubes.OpenURL:
	file.managed:
	- user: root
	- group: wheel
	- mode: 444
	- contents: \|
	@anyvm @anyvm ask,default_target=WWW
	```

	Add this to `/srv/salt/fedora.sls` to create the desktop file in the
	template:

	```yaml
	/usr/share/applications/browser_vm.desktop:
	file.managed:
	- user: root
	- group: wheel
	- mode: 444
	- contents: \|
	[Desktop Entry]
	Encoding=UTF-8
	Name=BrowserVM
	Exec=qvm-open-in-vm browser %u
	Terminal=false
	X-MultipleArgs=false
	Type=Application
	Categories=Network;WebBrowser;
	MimeType=x-scheme-handler/unknown;x-scheme-handler/about;text/html;text…
	```

	Create `/srv/salt/default_www.sls` with the following content, this
	will run xdg-settings to set the default browser:

	```yaml
	xdg-settings set default-web-browser browser_vm.desktop:
	cmd.run:
	- runas: user
	```

	Now, run `qubesctl --target fedora-38,dom0 state.apply`.

	From there, you MUST reboot the VMs that will be configured to use the
	WWW AppVm as the default browser, they need to have the new file
	`browser_vm.desktop` available for `xdg-settings` to succeed. Run
	`qubesctl --target vault,qubes-communication,qubes-devel state.apply`.

	Congratulations, now you will have a RPC prompt when an AppVM wants to
	open a file to ask you if you want to open it in your browsing AppVM.

	# Conclusion

	This method is a powerful way to handle your hosts, and it's ready to
	use on Qubes OS. Unfortunately, I still need to figure a nicer way to
	export the custom files written in /srv/salt/ and track the changes
	properly in a version control system.

	Erratum: I found a solution to manage the files :-) stay tuned for the
	next article.