Title: Nvidia card in eGPU and NixOS | |
Author: Solène | |
Date: 05 December 2021 | |
Tags: linux games nixos egpu | |
Description: How to setup an eGPU with NixOS using offloading prime | |
nvidia feature | |
# Updates | |
* 2022-01-02: add entry about specialization and how to use the eGPU as | |
a display device | |
# Introduction | |
I previously wrote about using an eGPU on Gentoo Linux. It was working | |
when using the eGPU display but I never got it to work for accelerating | |
games using the laptop display. | |
Now, I'm back on NixOS and I got it to work! | |
# What is it about? | |
My laptop has a thunderbolt connector and I'm using a Razer Core X | |
external GPU case that is connected to the laptop using a thunderbolt | |
cable. This allows to use an external "real" GPU on a laptop but it | |
has performance trade off and on Linux also compatibility issues. | |
There are three ways to use the nvidia eGPU: | |
- run the nvidia driver and use it as a normal card with its own | |
display connected to the GPU, not always practical with a laptop | |
- use optirun / primerun to run programs within a virtual X server on | |
that GPU and then display it on the X server (very clunky, originally | |
created for Nvidia Optimus laptop) | |
- use Nvidia offloading module (it seems recent and I learned about it | |
very recently) | |
The first case is easy, just install nvidia driver and use the right | |
card, it should work on any setup. This is the setup giving best | |
performance. | |
The most complicated setup is to use the eGPU to render what's | |
displayed on the laptop, meaning the video signal has to come back from | |
the thunderbolt cable, reducing the bandwidth. | |
# Nvidia offloading | |
Nvidia made work in their proprietary driver to allow a program to have | |
its OpenGL/Vulkan calls to be done in a GPU that is not the one used | |
for the display. This allows to throw optirun/primerun for this use | |
case, which is good because they added performance penalty, complicated | |
setup and many problems. | |
Official documentation about offloading with nvidia driver | |
# NixOS | |
I really love NixOS and for writing articles it's so awesome, because | |
instead of a set of instructions depending on conditions, I only have | |
to share the piece of config required. | |
This is the bits to add to your /etc/nixos/configuration.nix file and | |
then rebuild system: | |
```nixos configuration | |
hardware.nvidia.modesetting.enable = true; | |
hardware.nvidia.prime.sync.allowExternalGpu = true; | |
hardware.nvidia.prime.offload.enable = true; | |
hardware.nvidia.prime.nvidiaBusId = "PCI:10:0:0"; | |
hardware.nvidia.prime.intelBusId = "PCI:0:2:0"; | |
services.xserver.videoDrivers = ["nvidia" ]; | |
``` | |
A few notes about the previous chunk of config: | |
- only add nvidia to the list of video drivers, at first I was adding | |
modesetting but this was creating troubles | |
- the PCI bus ID can be found with lspci, it has to be translated in | |
decimal, here my nvidia id is 10:0:0 but in lspci it's 0a:00:00 with 0a | |
being 10 in hexadecimal | |
NixOS wiki about nvidia offload mode | |
# How to use it | |
The use of offloading is controlled by environment variables. What's | |
pretty cool is that if you didn't connect the eGPU, it will still work | |
(with integrated GPU). | |
## Running a command | |
We can use glxinfo to be sure it's working, add the environment as a | |
prefix: | |
```shell | |
__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxinfo | |
``` | |
## In Steam | |
Modify the command line of each game you want to run with the eGPU | |
(it's tedious), by: | |
```command line | |
__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia %command% | |
``` | |
## In Lutris | |
Lutris has a per-game or per-runner setting named "Enable Nvidia | |
offloading", you just have to enable it. | |
# Advanced usage / boot specialisation | |
Previously I only explained how to use the laptop screen and the eGPU | |
as a discrete GPU (not doing display). For some reasons, I've | |
struggled a LOT to be able to use the eGPU display (which gives more | |
performance because it's hitting less thunderbolt limitations). | |
I've discovered NixOS "specialisation" feature, allowing to add an | |
alternative boot entry to start the system with slight changes, in this | |
case, this will create a new "external-display" entry for using the | |
eGPU as the primary display device: | |
``` | |
hardware.nvidia.modesetting.enable = true; | |
hardware.nvidia.prime.sync.allowExternalGpu = true; | |
hardware.nvidia.prime.offload.enable = true; | |
hardware.nvidia.prime.nvidiaBusId = "PCI:10:0:0"; | |
hardware.nvidia.prime.intelBusId = "PCI:0:2:0"; | |
services.xserver.videoDrivers = ["nvidia" ]; | |
# external display on the eGPU card | |
# otherwise it's discrete mode using laptop screen | |
specialisation = { | |
external-display.configuration = { | |
system.nixos.tags = [ "external-display" ]; | |
hardware.nvidia.modesetting.enable = pkgs.lib.mkForce false; | |
hardware.nvidia.prime.offload.enable = pkgs.lib.mkForce false; | |
hardware.nvidia.powerManagement.enable = pkgs.lib.mkForce false; | |
services.xserver.config = pkgs.lib.mkOverride 0 | |
'' | |
Section "Module" | |
Load "modesetting" | |
EndSection | |
Section "Device" | |
Identifier "Device0" | |
Driver "nvidia" | |
BusID "10:0:0" | |
Option "AllowEmptyInitialConfiguration" | |
Option "AllowExternalGpus" "True" | |
EndSection | |
''; | |
}; | |
}; | |
``` | |
With this setup, the default boot is the offloading mode but I can | |
choose "external-display" to use my nvidia card and the screen attached | |
to it, it's very convenient. | |
I had to force the xserver configuration file because the one built by | |
NixOS was not working for me. |