Title: Faster packages updates with OpenBSD | |
Author: Solène | |
Date: 06 August 2021 | |
Tags: openbsd | |
Description: | |
# Introduction | |
On OpenBSD, pkg_add is not the fastest package manager around but it is | |
possible to make a simple change to make yours regular updates check | |
faster. | |
Disclaimer: THIS DOES NOT WORK ON -current/development version! | |
# Explanation | |
When you configure the mirror url in /etc/installurl, on release/stable | |
installations when you use "pkg_add", some magic happens to expand the | |
base url into full paths usable by PKG_PATH. | |
```shell | |
http://ftp.fr.openbsd.org/pub/OpenBSD | |
becomes | |
http://ftp.fr.openbsd.org/pub/OpenBSD/%v/packages-stable/%a/:http://ftp.fr.open… | |
``` | |
The built string passed to PKG_PATH is the concatenation (joined by a | |
":" character) of the URL toward /packages/ and /packages-stable/ | |
directories for your OpenBSD version and architecture. | |
This is why when you use "pkg_info -Q foobar" to search for a package | |
and that a package name matches "foobar" in /packages-stable/ pkg_info | |
will stop, it search for a result in the first URL given by PKG_PATH, | |
when you add -a like "pkg_info -aQ foobar", it will look in all URL | |
available in PKG_PATH. | |
# Why we can remove /packages/ | |
When you run your OpenBSD system freshly installed or after an upgrade, | |
when you have your packages sets installed from the repository of your | |
version, the files in /packages/ in the mirrors will NEVER CHANGE. | |
When you run "pkg_add -u", it's absolutely 100% sure nothing changed in | |
the directory /packages/, so checking for changes against them every | |
time make no sense. | |
Using "pkg_add -u" with the defaults makes sense when you upgrade from | |
a previous OpenBSD version because you need to upgrade all your | |
packages. But then, when you look for security updates, you only need | |
to check against /packages-stable/. | |
# How to proceed | |
There are two ways, one reusing your /etc/installurl file and the other | |
is hard coding it. Pick the one you prefer. | |
```shell commands | |
# reusing the content of /etc/installurl | |
env PKG_PATH="$(cat /etc/installurl)/%v/packages-stable/%a/" pkg_add -u | |
# hard coding the url | |
env PKG_PATH="http://ftp.fr.openbsd.org/pub/OpenBSD/%v/packages-stable/%a/" pkg… | |
``` | |
Be careful, you will certainly have a message like this: | |
```text | |
Couldn't find updates for ImageMagick-6.9.12.2 adwaita-icon-theme-3.38.0 aom-2.… | |
``` | |
This is perfectly normal, as pkg_add didn't find the packages in | |
/packages-stable/ it wasn't able to find the current version installed | |
or an update, as we only want updates it's fine. | |
# Simple benchmark | |
On my server running 6.9 with 438 packages I get these results. | |
* packages-stable only: 44 seconds | |
* all the packages: 203 seconds | |
I didn't measure the bandwidth usage but it should scale with the time | |
reduction. | |
# Conclusion | |
This is a very simple and reliable way to reduce the time and bandwidth | |
required to check for updates on OpenBSD (non -current!). I wonder if | |
this would be a good idea to provide it as a flag for pkg_add, like | |
"only check for stable updates". |