Title: My NixOS workflow after migrating from OpenBSD | |
Author: Solène | |
Date: 24 September 2022 | |
Tags: openbsd nixos life | |
Description: This article tells my journey migrating my computers from | |
OpenBSD to NixOS | |
# Introduction | |
After successfully switching my small computer fleet to NixOS, I'd like | |
to share about the journey. | |
I currently have a bunch of computers running NixOS: | |
* my personal laptop | |
* the work laptop | |
* home router | |
* home file server | |
* some home lab computer | |
* e-mail / XMPP / Gemini server hosted at openbsd.amsterdam | |
That sums up to 6 computers running NixOS, half of them is running the | |
development version, and the other is running the latest release. | |
# Migration | |
## From OpenBSD to NixOS | |
All the computers above used to run OpenBSD, let me explain why I | |
migrated. It was a very complicated choice for me, because I still | |
like OpenBSD despite I uninstalled it. | |
* NixOS offers more software choice than OpenBSD, this is especially | |
true for recent software, and porting them to OpenBSD is getting | |
difficult over time. | |
* After spending too much time with OpenBSD, I wanted to explore a | |
whole new world, NixOS being super different, it was a good | |
opportunity. As a professional IT worker, it's important for me to | |
stay up to date, Linux ecosystem evolved a lot over that past ten | |
years. What's funny is OpenBSD and NixOS share similar issues such as | |
not being able to use binaries found on the Internet (but for various | |
reasons) | |
* NixOS maintenance is drastically reduced compared to OpenBSD | |
* NixOS helps me to squeeze more from my hardware (speed, storage | |
capacity, reliability) | |
* systemd: I bet this one will be controversial, but since I learned | |
how to use it, I really like it (and NixOS make it even greater for | |
writing units) | |
Security is hard to measure, but it's the main argument in favor of | |
OpenBSD, however it is possible to enable mitigations on Linux as well | |
such as hardened memory allocator or a hardened Kernel. OpenBSD isn't | |
practical to separate services from running all in the same system, | |
while on Linux you can easily sandbox services. In the end, the | |
security mechanisms are different, but I feel the result is pretty | |
similar for my threat model of protecting against script kiddies. | |
I give a bonus point for Linux for its ability to account | |
CPU/Memory/Swap/Disk/network per user, group and process. This allows | |
spotting unusual activity. Security is about protection, but also | |
about being aware of intrusion, OpenBSD isn't very good at it at the | |
moment. | |
## NixOS modules | |
One issue I had migrating my mail server and the router was to find | |
what changes were made in /etc. I was able to figure which services | |
were enabled, but not really all the steps done a few years ago to | |
configure them. I had to scrape all the configuration file to see if | |
it looked like verbatim default configuration or something I changed | |
manually. | |
This is where NixOS shines for maintenance and configuration, | |
everything is declarative, so you never touch anything in /etc. At | |
anytime, even in a few years, I'll be able to exactly tell what I need | |
for each service, without having to dig up /etc/ and compare with | |
default files. This is a saner approach, and also ease migration | |
toward another system (OpenBSD? ;) ) because I'd just have to apply | |
these changes to configuration files. | |
# Workflow | |
Working with NixOS can be disappointing. Most of the system is | |
read-only, you need to learn a new language (Nix) to configure | |
services, you have to "rebuild" your system to make a change as simple | |
as adding an entry in /etc/hosts, not very "Unix-like". | |
Your biggest friend is the man page configuration.nix which contains | |
all the possible configurations settings available in NixOS, from | |
Kernel choice and grub parameters, to Docker containers started at boot | |
or your desktop environment. | |
The workflow is pretty easy, take your configuration.nix file, apply | |
changes to it, and run "nixos-rebuild test" (or switch if you prefer) | |
to try the changes. Then, you may want something more elaborated like | |
tracking your changes in a git or darcs repository, and start sharing | |
pieces of configuration between machines. | |
But in the end, you just declare some configuration. I prefer to keep | |
my configurations very easy to read, I still don't have any modules or | |
much variable, the common pieces are just .nix files imported for the | |
systems needing it. It's super easy to track and debug. | |
# Bento | |
Bento GitHub project page | |
After a while, I found it very tedious to have to run nixos-rebuild on | |
each machine to keep them up to date, so I started using the | |
autoUpgrade module which basically do it for you in a scheduled task. | |
But then, I needed to centralize each configuration file somewhere, and | |
have fun with ssh keys because I don't like publishing my configuration | |
files publicly. Which isn't optimal either as if you make a change | |
locally, you need to push the changes and connect to the remote host to | |
pull the changes and rebuild immediately instead of waiting for the | |
auto upgrade process. | |
So, I wrote bento, which allows me to manage all the configuration | |
files in a single place, but better than that, I can build the | |
configuration locally to ensure they will work once shipped. I quickly | |
added a way to track the status of each remote system to be sure they | |
picked up and applied the changes (every 10 minutes). Later, I | |
improved the network efficiency by central management computer as a | |
local binary cache, so other systems are now downloading packages from | |
it locally, instead of downloading them again from the Internet. | |
The coolest thing ever is that I can manage offline systems such as my | |
work laptop, I can update its configuration file in the weekend for an | |
update or to improve the environment (it mostly shares the same | |
configuration as my main laptop), and it will automatically pick it up | |
when I boot it. | |
# Conclusion | |
Moving to NixOS was a very good and pleasant experience, but I had some | |
knowledge about it before starting. It might be confusing a lot of | |
people, and you certainly need to get into NixOS mindset to appreciate | |
it. |