Title: How to make a local NixOS cache server

	Title: How to make a local NixOS cache server
	Author: Solène
	Date: 02 June 2022
	Tags: nixos unix bandwidth nocloud
	Description: This explains how to create your own local cache for NixOS
	packages in order to save bandwidth.

	# Introduction

	If like me, you have multiple NixOS system behind the same router, you
	may want to have a local shared cache to avoid downloading packages
	multiple time.

	This can be done simply by using nginx as a reverse proxy toward the
	official repository and by enabling caching the result.

	nix-binary-cache-proxy project I used as a base

	# Server side configuration

	We will declare a nginx service on the server, using http protocol only
	to make setup easier. The packages are signed, so their authenticity
	can't be faked. In this setup, using https would add anonymity which
	is not much of a concern in a local network, for my use case.

	In the following setup, the LAN cache server will be reachable at the
	address 10.42.42.150, and will be using the DNS resolver 10.42.42.42
	every time it needs to reach the upstream server.

	```nix code
	services.nginx = {
	enable = true;
	appendHttpConfig = ''
	proxy_cache_path /tmp/pkgcache levels=1:2 keys_zone=cachecache:100m max_s…

	# Cache only success status codes; in particular we don't want to cache 4…
	# See https://serverfault.com/a/690258/128321
	map $status $cache_header {
	200 "public";
	302 "public";
	default "no-cache";
	}
	access_log /var/log/nginx/access.log;
	'';

	virtualHosts."10.42.42.150" = {
	locations."/" = {
	root = "/var/public-nix-cache";
	extraConfig = ''
	expires max;
	add_header Cache-Control $cache_header always;
	# Ask the upstream server if a file isn't available locally
	error_page 404 = @fallback;
	'';
	};

	extraConfig = ''
	# Using a variable for the upstream endpoint to ensure that it is
	# resolved at runtime as opposed to once when the config file is loaded
	# and then cached forever (we don't want that):
	# see https://tenzer.dk/nginx-with-dynamic-upstreams/
	# This fixes errors like
	# nginx: [emerg] host not found in upstream "upstream.example.com"
	# when the upstream host is not reachable for a short time when
	# nginx is started.
	resolver 10.42.42.42;
	set $upstream_endpoint http://cache.nixos.org;
	'';

	locations."@fallback" = {
	proxyPass = "$upstream_endpoint";
	extraConfig = ''
	proxy_cache cachecache;
	proxy_cache_valid 200 302 60d;
	expires max;
	add_header Cache-Control $cache_header always;
	'';
	};

	# We always want to copy cache.nixos.org's nix-cache-info file,
	# and ignore our own, because `nix-push` by default generates one
	# without `Priority` field, and thus that file by default has priority
	# 50 (compared to cache.nixos.org's `Priority: 40`), which will make
	# download clients prefer `cache.nixos.org` over our binary cache.
	locations."= /nix-cache-info" = {
	# Note: This is duplicated with the `@fallback` above,
	# would be nicer if we could redirect to the @fallback instead.
	proxyPass = "$upstream_endpoint";
	extraConfig = ''
	proxy_cache cachecache;
	proxy_cache_valid 200 302 60d;
	expires max;
	add_header Cache-Control $cache_header always;
	'';
	};
	};
	};
	```

	Be careful, the default cache is located under /tmp/ but the nginx
	systemd service is hardened and its /tmp/ is faked in a temporary
	directory, meaning if you restart nginx you lose the cache. I'd advise
	using a directory like /var/cache/nginx/ if you want your cache to
	persist across restarts.

	# Client side configuration

	Using the cache server on a system is really easy. We will define the
	binary cache to our new local server, the official cache is silently
	added so we don't have to list it.

	```nix code
	nix.binaryCaches = [ "http://10.42.42.150/" ];
	```

	Note that you have to use this on the cache server itself if you want
	the system to use the cache for its own needs.

	# Conclusion

	Using a local cache can save a lot of bandwidth when you have more than
	one computer at home (or if you extensively use nix-shell and often run
	the garbage collector). Due to NixOS packages names being unique, we
	won't have any issues of a newer package version behind hidden by a
	local copy cached, which make the setup really easy.