Title: Lessons learned with XZ vulnerability

	Title: Lessons learned with XZ vulnerability
	Author: Solène
	Date: 30 March 2024
	Tags: security linux openbsd
	Description: In this article I share my thoughts about the recent xz
	vulnerability disclosure and what we could do to cope with it

	# Intro

	Yesterday Red Hat announced that xz library was compromised badly, and
	could be use as a remote execution code vector. It's still not clear
	exactly what's going on, but you can learn about this on the following
	GitHub discussion that also links to original posts:

	Discussion about xz being compromised

	# What's the state?

	As far as we currently know, xz-5.6.0 and xz-5.6.1 contains some really
	obfsucated code that would trigger only in sshd, this only happen in
	the case of:

	* the system is running systemd
	* openssh is compiled with a patch to add a feature related to systemd
	* the system is using glibc (this is mandatory for systemd systems
	afaik anyway)
	* xz package was built using release tarballs published on GitHub and
	not auto-generated tarballs, the malicious code is missing in the git
	repository

	So far, it seems openSUSE Tumbleweed, Fedora 40 and 41 and Debian sid
	were affected and vulnerable. Nobody knows what the vulnerability is
	doing exactly yet, when security researchers get their hands on it, we
	will know more.

	OpenBSD, FreeBSD, NixOS and Qubes OS (dom0 + official templates) are
	unaffected. I didn't check for other but Alpine and Guix shouldn't be
	vulnerable either.

	Gentoo security advisory (unaffected)

	# What lessons could we learn?

	This is really unfortunate that a piece of software as important and
	harmless in appareance got compromised. This made me think about how
	could we protect the most against this kind of issues, I came to the
	conclusion:

	* packages should be built from source code repository instead of
	tarballs whenever possible (sometimes tarballs contain vendoring code
	which would be cumbersome to pull otherwise), at least we would know
	what to expect
	* public network services that should be only used by known users (like
	openssh, imap server in small companies etc..) should be run behind a
	VPN
	* OpenBSD style to have a base system developed as a whole by a single
	team is great, such kind of vulnerability is barely possible to happen
	(on base system only, ports aren't audited)
	* whenever possible, separate each network service within their own
	operating system instance (using hardware machines, virtual machines or
	even containers)
	* avoid daemons running as root as possible
	* use opensnitch on workstations (linux only)
	* control outgoing traffic whenever you can afford to

	I don't have much opinion about what could be done to protect supply
	chain. As a packager, it's not possible to audit code of each software
	we update. My take on this is we have to deal with it, xz may
	certainly not be the only one vulnerable library running in production.

	However, the risks could be reduced by:

	* using less programs
	* using less complex programs
	* compiling programs with less options to pull in less dependencies
	(FreeBSD and Gentoo both provide this feature and it's great)

	# Conclusion

	I actually have two systems that were running the vulnerable libs on
	openSUSE MicroOS which updates very aggressively (daily update + daily
	reboot). There are no magic balance between "update as soon as
	possible" and "wait for some people to take the risks first".

	I'm going to rework my infrastructure and expose the bare minimum to
	the Internet, and use a VPN for all my services that are for known
	users. The peace of mind will obtained be far greater than the burden
	of setting up WireGuard VPNs.