Title: Benchmarking compilation time with ccache/mfs on OpenBSD | |
Author: Solène | |
Date: 18 September 2021 | |
Tags: openbsd benchmark | |
Description: | |
# Introduction | |
I always wondered how to make packages building faster. There are at | |
least two easy tricks available: storing temporary data into RAM and | |
caching build objects. | |
Caching build objects can be done with ccache, it will intercept cc and | |
c++ calls (the programs compiling C/C++ files) and depending on the | |
inputs will reuse a previously built object if available or build | |
normally and store the result for potential next reuse. It has nearly | |
no use when you build software only once because it requires objects to | |
be cached before being useful. It obviously doesn't work for non C/C++ | |
programs. | |
The other trick is using a temporary filesystem stored in memory (RAM), | |
on OpenBSD we will use mfs but on Linux or FreeBSD you could use tmpfs. | |
The difference between those two is mfs will reserve the given memory | |
usage while tmpfs is faster and won't reserve the memory of its | |
filesystem (which has pros and cons). | |
So, I decided to measure the build time of the Gemini browser Lagrange | |
in three cases: without ccache, with ccache but first build so it | |
doesn't have any cached objects and with ccache with objects in it. I | |
did these three tests multiple time because I also wanted to measure | |
the impact of using memory base filesystem or the old spinning disk | |
drive in my computer, this made a lot of tests because I tried with | |
ccache on mfs and package build objects (later referenced as pobj) on | |
mfs, then one on hdd and the other on mfs and so on. | |
To proceed, I compiled net/lagrange using dpb after cleaning the | |
lagrange package generated everytime. Using dpb made measurement a lot | |
easier and the setup was reliable. It added some overhead when | |
checking dependencies (that were already installed in the chroot) but | |
the point was to compare the time difference between various tweaks. | |
# Results numbers | |
Here are the results, raw and with a graphical view. I did run | |
multiples time the same test sometimes to see if the result dispersion | |
was huge, but it was reliable at +/- 1 second. | |
```Raw results | |
Type Duration for second build Duration with empt… | |
ccache mfs + pobj mfs 60 133 | |
ccache mfs + pobj hdd 63 130 | |
ccache hdd + pobj mfs 61 127 | |
ccache hdd + pobj hdd 68 137 | |
no ccache + pobj mfs 124 | |
no ccache + pobj hdd 128 | |
``` | |
Diagram with results | |
# Results analysis | |
At first glance, we can see that not using ccache results in builds a | |
bit faster, so ccache definitely has a very small performance impact | |
when there is no cached objects. | |
Then, we can see results are really tied together, except for the | |
ccache and pobj both on the hdd which is the slowest combination by far | |
compared to the others times differences. | |
# Problems encountered | |
My building system has 16 GB of memory and 4 cores, I want builds to be | |
as fast as possible so I use the 4 cores, for some programs using Rust | |
for compilation (like Firefox), more than 8 GB of memory (4x 2GB) is | |
required because of Rust and I need to keep a lot of memory available. | |
I tried to build it once with 10GB of mfs filesystem but when packaging | |
it did reach the filesystem limit and fail, it also swapped during the | |
build process. | |
When using a 8GB mfs for pobj, I've been hitting the limit which | |
induced build failures, building four ports in parallel can take some | |
disk space, especially at package time when it copies the result. It's | |
not always easy to store everything in memory. | |
I decided to go with a 3 GB ccache over MFS and keep the pobj on the | |
hdd. | |
I had no spare SSD to add an SSD to the list. :( | |
# Conclusion | |
Using mfs for at least ccache or pobj but not necessarily both is | |
beneficial. I would recommend using ccache in mfs because the memory | |
required to store it is only 1 or 2 GB for regular builds while storing | |
the pobj in mfs could requires a few dozen gigabytes of memory (I think | |
chromium requires 30 or 40 GB last time I tried). |