Title: External GPU on Linux review

	Title: External GPU on Linux review
	Author: Solène
	Date: 01 September 2021
	Tags: linux gentoo games egpu
	Description:

	# Introduction

	I like playing video games, and most games I play require a GPU that is
	more powerful than the integrated graphic chipset that can be found in
	laptop or computers. I recently found that external graphic card were
	a thing, and fortunately I had a few spare old graphic card for trying.

	The hardware is called an eGPU (for external GPU) and are connected to
	the computer using a thunderbolt link. Because I buy most of my
	hardware second hand now, I've been able to find a Razer Core X eGPU
	(the simple core X and not the core X Chroma which provides USB and
	RJ45 connectivity on the case through thunderbolt), exactly what I was
	looking for. Basically, it's an external case with a PSU inside and a
	rack, pull out the rack and insert the graphic card, and you are done.
	Obviously, it works fine on Windows or Mac but it can be tricky on
	Linux.

	Razer core X product

	Attempt to make a picture of my eGPU with an nvidia 1060 in it

	# My setup

	I'm using a Lenovo T470 with an i5 CPU. When I want to use the eGPU, I
	connect the thunderbolt wire and keyboard / mouse (which I connect
	through an USB KVM to switch those from a computer to another). The
	thunderbolt port also provide power to the laptop which is good to
	know.

	# How does it work?

	There are two ways to use this device, the display can be connected to
	the eGPU itself or the rendering could be done on the laptop (let's say
	we only target laptops here) using the eGPU as a discrete card (only
	rendering, without display). Both modes have pros and cons.

	* External display Pros: best performance, allow many displays to be
	used
	* External display Cons: require a screen

	* Discrete mode Pros: no extra wire, no different setup when using the
	laptop without the eGPU
	* Discrete mode Cons: performance penalty, support doesn't work well on
	Linux

	The performance penalty comes from the fact the thunderbolt bandwidth
	is limited, and if you want to display on your screen you need to
	receive the data back which will reduce the bandwidth allowed for
	rendering. A penalty of at least 20% should be expected in normal
	mode, and around 40% in discrete mode. This is not really fun but for
	a nice boost with an old graphic card this is still nice.

	eGPU on Linux with a Razer core X Chroma
	eGPU benchmarks

	# Configuration (NVIDIA)

	It's quite simple now in 2023 (blog update), the first step is to
	install nvidia-drivers.

	## Discrete mode

	- Add your user to the video group (at least on Gentoo)
	- No `/etc/X11/xorg.conf` file is required
	- The graphical card should appear in `nvidia-settings` under a "PRIME"
	menu
	- Use `prime-run` as a prefix to run commands, the discrete mode is
	simply enabled by environment variables. If `prime-run` isn't a thing
	in your distribution, create a script `nvidia-offload` like explained
	in the NixOS wiki

	NixOS wiki: Nvidia - offload mode

	If you want to run Flatpak programs with the discrete GPU, you will
	need to set all the environment variables in the flatpak program
	environment. You can't just set them in your shell and run flatpak
	from there because of the sandboxing.

	## External display

	- Run `nvidia-xconfig` to create a `/etc/X11/xorg.conf` file that uses
	the Nvidia card as the main display

	## Both mode

	I ended figuring a `xorg.conf` allowing me to keep the same file with
	and without the eGPU, and to use the discrete and external display at
	the same time. The funniest part is if you run a program on the nvidia
	screen and move it back to the laptop screen, the eGPU continues to
	render it.

	It's by far the most convenient configuration as you have nothing to
	tweak, and you can use laptop + eGPU displays.

	```xorg.conf
	Section "ServerLayout"
	Identifier "layout"
	Screen 0 "intel"
	Inactive "nvidia"
	Option "AllowNVIDIAGPUScreens"
	EndSection

	Section "Device"
	Identifier "intel"
	Driver "modesetting"
	BusID "PCI:0:2:0"
	EndSection

	Section "Screen"
	Identifier "intel"
	Device "intel"
	EndSection

	Section "Device"
	Identifier "nvidia"
	Driver "nvidia"
	BusID "PCI:10:0:0"
	Option "AllowExternalGpus" "True"
	EndSection

	Section "Screen"
	Identifier "nvidia"
	Device "nvidia"
	EndSection
	```



	## Switching between modes

	If you want to switch from one to the other, you need to exit all X
	servers first. Booting with a `xorg.conf` for Nvidia while not having
	a Nvidia card plugged in will prevent X to start, which is annoying.

	The program egpu-switch can help in that regard, but it can't choose
	between discrete or external display mode, you will need to decide
	which mode you prefer when the card is plugged by providing the
	according xorg.conf file.

	egpu-switcher GitHub project page

	# What to expect of it on Linux?

	I've been using this on Gentoo only so far, but I had a previous
	experience with a pretty similar setup a few years ago with a laptop
	with a discrete nvidia card (called Optimus at that time), and the GPU
	was only usable as a discrete GPU and it was a mess at that time.

	As for the eGPU, in external mode it works fine using the nvidia
	driver, I needed an xorg.conf file to tell to use the nvidia driver,
	then the display would be fine and 3D would work perfectly as if I was
	using a "real" card on a computer. I can play high demanding games
	such as Control, Death Stranding or other games using my Thinkpad
	Laptop when docked, this is really nice!

	The setup is a bit weird though, if I want to undock, I need to prepare
	the new xorg.conf file and stop X, disconnect the eGPU and restart the
	display manager to login. Not very easy. I've been able to script it
	using a simple script at boot that will detect the Nvidia GPU and
	choose the correct xorg.conf file just before starting the display
	manager, it works quite fine and makes life easier.

	# Video games?

	I've been playing Steam video games, it works absolutely perfectly due
	to their work on Proton to make Windows games running. GOG games works
	fine too, I use Lutris games library manager to handle them and it
	works so far.

	Now, there is the tricky discrete mode. On linux, the bumblebee
	project allows rendering a program in a virtual display to benefit from
	the 3D acceleration and then show it on another device, this work was
	done for Optimus hardware hence the bumblebee name (related to
	Transfomers lore). Steam doesn't like bumblebee at all and won't start
	game, this is a known bug, Steam is bad at managing multiple GPUs.
	I've not been able to display anything using bumblebee.

	On the other hand, native Linux GOG games were working fine using
	bumblebee, however I don't own much high demanding Linux games so I've
	not been able to see if the performance hit was hard. Windows GOG
	games wouldn't run, partially because the DXVK (directX to vulkan) Wine
	rendering can't be used because bumblebee doesn't allow using Vulkan
	graphical API and error messages were unhelpful. I have literally lost
	two days of my life trying to achieve something useful with the
	discrete GPU mode but nothing came out of it, except native Linux
	games.

	Playing Control on Gentoo (windowed for the screen)

	# Why using an eGPU?

	Laptops are very limited in their upgrade capabilities, adding a GPU
	could avoid someone to own a "gaming" tower PC and a good laptop. The
	GPU is 100% replaceable because the case offers a pci express port and
	a standard PSU (which can be replaced too!). The EGPU could be shared
	among a few users in a home too. This is a nice way to recycling old
	GPUs for a nice graphic boost to play everything that is more than 5
	years old (and that's a bunch of good games!). I think using a top
	notch GPU in this would be a waste though.

	# Conclusion

	I'm pretty happy with the experience so far, now I can play my
	favorites games on Linux using the same computer I like to use all the
	day. While the experience is not as plug and play than it is on
	Windows, it is solid and stable.

	# Troubleshoot

	Some reminders when something is wrong.

	## OpenSUSE setup

	This distribution ships with a tool "prime-select" which is very
	convenient, you can pick which driver you want to enable first, or if
	you want to do discrete rendering.

	## No sound

	An udev rule is certainly blocking the audio device for some reasons...
	On a system, I found the file
	"/lib/udev/rules.d/90-nvidia-udev-pm-G05.rules" with a comment about
	disabling audio devices, commenting it solves the problem.