Title: Using GitHub Actions to maintain Gentoo packages repository | |
Author: Solène | |
Date: 04 March 2023 | |
Tags: gentoo automation | |
Description: In this article, I explain how I used GitHub actions to | |
build a packages repository for Gentoo and keep it up to date | |
# Introduction | |
In this blog post, I'd like to share how I had fun using GitHub actions | |
in order to maintain a repository of generic x86-64 Gentoo packages up | |
to date. | |
Built packages are available at https://interbus.perso.pw/ and can be | |
used in your `binrepos.conf` for a generic x86-64 packages provider, | |
it's not building many packages at the moment, but I'm open to add more | |
packages if you want to use the repository. | |
GitHub Project page: Build Gentoo Packages For Me | |
# Why | |
I don't really like GitHub, but if we can use their CPU for free for | |
something useful, why not? The whole implementation and setup looked | |
fun enough that I should give it a try. | |
I was using a similar setup locally to build packages for my Gentoo | |
netbook using a more powerful computer, so it was actually achievable, | |
so I had to try. I don't have much use of it myself, but maybe a | |
reader will enjoy the setup and do something similar (maybe not for | |
Gentoo). | |
My personal infrastructure is quite light, with only an APU router plus | |
a small box with an Atom CPU as a NAS, I was looking for a cheap way to | |
keep their Gentoo systems running without having to compile locally. | |
# Challenges | |
Building a generic Gentoo packages repository isn't straighforward for | |
a rew reasons: | |
* compilation flags must match all the consumers' architecture | |
* default USE flags must be useful for many | |
* no support for remote builders | |
* the whole repository must be generated on a single machine with all | |
the files (can't be incremental) | |
Fortunately, there are Gentoo containers images that can be used to | |
start a fresh Gentoo, and from there, build packages from a clean | |
system every time. Packages have to be added into the container before | |
each change, otherwise the file `Packages` that will be generated as a | |
repository index won't contain all the files. | |
Using a `-march=x86-64` compiler flag allows targeting all the amd64 | |
systems, at the cost of less optimized binaries. | |
For the USE flags, a big part of Gentoo, I chose to select a default | |
profile and simply stick with it. People using the repository could | |
still change their USE flags, and only pick the binary packages from | |
the repo if they still match expectations. | |
# Setup | |
We will use GitHub actions (Free plan) to build packages for a given | |
Gentoo profile, and then upload it to a remote server that will share | |
the packages over HTTPS. | |
The plan is to use a docker image of a stage3 Gentoo provided by the | |
project gentoo-docker-images, pull previously built packages from my | |
server, build new packages or updating existing packages, and push the | |
changes to my server. Meanwhile, my server is serving the packages | |
over https. | |
GitHub's actions are a feature from GitHub allowing to create | |
Continuous Integration easy by providing "actions" (reusable components | |
made by other) that you organize in steps. | |
For the job, I used the following steps on an Ubuntu system: | |
1. Deploy SSH keys (used to pull/push packages to my server) stored as | |
secrets in the GitHub project | |
2. Checkout the sources of the project | |
3. Make a local copy of the packages repository | |
4. Create a container image based on the Gentoo stage3 + instructions | |
to run | |
5. Run the image that will use emerge to build the packages | |
6. Copy the new repository on the remote server (using rsync to copy | |
the diff) | |
GitHub project page: Gentoo Docker Images | |
# Problems encountered | |
While the idea is simple, I faced a lot of build failures, here is a | |
list of problems I remember. | |
## Go is failing to build (problem is Docker specific) | |
For some reasons, Go was failing to build with a weird error, this is | |
due to some sandboxing done by emerge that wasn't allowed by the Docker | |
environment. | |
The solution is to loose the sandboxing with `FEATURES="-ipc-sandbox | |
-pid-sandbox -sandbox -usersandbox"` in `/etc/portage/make.conf`. | |
That's not great. | |
## Raw stage3 is missing pieces | |
The starter image is a stage3 of Gentoo, it's quite bare, one critical | |
package missing to build other but never pulled as dependency is kernel | |
sources. | |
You need to install `sys-kernel/gentoo-sources` if you want builds to | |
succeed for many packages. | |
## No merged-usr profile | |
The gentoo docker images repository isn't provided merged-usr profiles | |
(yet?), I had to install merged-usr and run it, to have a correct | |
environment matching the selected profile. | |
## Compilation is too long | |
The job time is limited to 6h00 on the free plan, I added a timeout for | |
the emerge doing the building job to stop a bit earlier, to let it some | |
time to push the packages to the remote server, this will allow saving | |
time for the next run. Of course, this only works until a single | |
package require more than the timeout time to build (but it's quite | |
unlikely given the CI is fast enough). | |
# Security | |
One has to trust GitHub actions, GitHub employees may have access to | |
jobs running there, and could potentially compromise built packages | |
using a rogue container image. While it's unlikely, this is a | |
possibility. | |
Also, please note that the current setup doesn't sign the packages. | |
This is something that could be added later, you can find documentation | |
on the Gentoo Wiki for this part. | |
Gentoo Wiki: Binary package guide | |
Another interesting area for security was the rsync access of the | |
GitHub actions to easily synchronize the packages with the builder. | |
It's possible to restrict an SSH key to a single command to run, like a | |
single rsync with no room to change a single parameter. Unfortunately, | |
the setup requires using rsync in two different cases: downloading and | |
pushing files, so I had to write a wrapper looking at the variable | |
`SSH_COMMAND` and allowing either the "pull" rsync, or the "push" | |
rsync. | |
Restrict rsync command over SSH | |
# Conclusion | |
The GitHub free plan allows you to run a builder 24/7 (with no parallel | |
execution), it's really fast enough to keep a non-desktop @world up to | |
date. If you have a pro account, the local cache GitHub cache may not | |
be limited, and you may be able to keep the built packages there, | |
removing the "pull packages" step. | |
If you really want to use this, I'd recommend using a schedule in the | |
GitHub action to run it every day. It's as simple as adding this in | |
the GitHub workflow. | |
```yaml | |
on: | |
schedule: | |
- cron: '0 2 * * *' # every day at 02h00 | |
``` | |
# Credits | |
I would like to thank Jonathan Tremesaygues who wrote most of the | |
GitHub actions pieces after I shared with him about my idea and how I | |
would implement it. | |
Jonathan Tremesaygues's website | |
# Going further | |
Here is a simple script I'm using to use a local Linux machine as a | |
Gentoo builder for the box you run it from. It's using a gentoo stage3 | |
docker image, populated with packages from the local system and its | |
`/etc/portage/` directory. | |
Note that you have to use `app-misc/resolve-march-native` to generate | |
the compiler command line parameters to replace `-march=native` because | |
you want the remote host to build with the correct flags and not its | |
own `-march=native`, you should also make sure those flags are working | |
on the remote system. From my experience, any remote builder newer | |
than your machine should be compatible. | |
Tildegit: Example of scripts to build packages on a remote machine for the loca… |