Title: Avoid Linux locking up in low memory situations using earlyoom | |
Author: Solène | |
Date: 28 September 2022 | |
Tags: linux nixos portoftheweek | |
Description: This article presents the program earlyoom to prevent a | |
Linux system to lock up in low memory situations. | |
# Introduction | |
Within operating system kernels, at least for Linux and the BSDs, there | |
is a mechanism called "out of memory killer" which is triggered when | |
the system is running out of memory and some room must be made to make | |
the system responsive again. | |
However, in practice this OOM mechanism doesn't work well. If the | |
system is running out of memory, it will become totally unresponsive, | |
and sometimes the OOM killer will help, but it may take like 30 | |
minutes, but sometimes it may be stuck forever. | |
Today, I stumbled upon a nice project called "earlyoom", which is an | |
OOM manager working in the user land instead of inside the kernel, | |
which gives it a lot more flexibility about its actions and the | |
consequences. | |
earlyoom GitHub project page | |
# How it works | |
earlyoom is simple in that it's a daemon running as root, using nearly | |
no memory, that will regularly poll for remaining swap memory and RAM | |
memory, if the current level are below the threshold of both, actions | |
will be taken. | |
What's cool is you can tell it to prefer some processes to terminate | |
first, and some processes to avoid as much as possible. For some | |
people, it may be preferable to terminate a web browser first and | |
instant messaging than their development software. | |
I use it with the following parameters: | |
```shell | |
earlyoom -m 2 -s 2 -r 3600 -g --avoid '^(X|plasma.*|konsole|kwin)$' --prefer '^… | |
``` | |
The command line above means that if my system has less than 2% of its | |
RAM and less than 2% of its swap available, earlyoom will try to | |
terminate existing program whose binary matches | |
electron/libreoffice/gimp etc.... and avoid programs named | |
X/Plasma.*/konsole/kwin. | |
For configuring it properly as a service, explanations can be found in | |
the project README file. | |
# NixOS setup | |
On NixOS, there is a module for earlyoom, to configure it like in the | |
example above: | |
```nix | |
{ | |
services.earlyoom = { | |
enable = true; | |
freeSwapThreshold = 2; | |
freeMemThreshold = 2; | |
extraArgs = [ | |
"-g" "--avoid '^(X|plasma.*|konsole|kwin)$'" | |
"--prefer '^(electron|libreoffice|gimp)$'" | |
]; | |
}; | |
} | |
``` | |
# Conclusion | |
This program is a pleasant surprise to me, I often run out of memory on | |
my laptop because I'm running some software requiring a lot of memory | |
for good reasons, and while the laptop has barely enough memory to run | |
them, I should have most of the other software close to make it fit in. | |
However, when I forget to close them, the system would just lock up | |
for a while, which most often require a hard reboot. Being able to | |
avoid this situation is a big plus for me. Of course, adding some swap | |
space would help, but I prefer to avoid adding more swap as it's | |
terribly inefficient and only postpone the problem. |