---------------------------------------- | |
lxd | |
January 28th, 2019 | |
---------------------------------------- | |
lxd and lxc are amazing. If you know what they are, skip this | |
paragraph. If not, they're sort of like Docker, but instead of | |
encapsulating your application, they encapsulate a system. They're | |
not as broad as a whole VM, and in reality they're not really | |
a "thing" at all. They're a collection of isolation tools built | |
nto Linux that makes it seem like you're running on a whole new | |
vm while sharing kernels and junk. Um, go read about it somewhere | |
else. There's some great material out there that will explain it | |
better than me. | |
The purpose if this phlog is to document my basic LXD setup. My | |
host system is Ubuntu 18.04 as of this article being written, but | |
t really doesn't matter. Most of my containers are built using | |
the latest stable Ubuntu, but that also doesn't really matter. My | |
goals are as follows: | |
- Be able to create and dispose of containers easily | |
- All containers, by default, allow me to ssh in using my keys | |
- I can ssh to the containers by their container name, not IP | |
- I can apply a 'gui' profile and make a container work with my | |
native display | |
- I can apply a 'user' profile and make my container start with my | |
dotfiles installed | |
Step 1: Install LXD | |
If it's not already installed, you can grab the snap package. | |
Step 2: Install ZFS | |
It's not necessary, but it performs better at start/stop | |
operations. If you install it, LXD will use it as the default. | |
Step 3: Configure LXD | |
$ sudo lxd init | |
Just say yes to the defaults. | |
Step 4: Edit the default profile | |
$ lxc profile edit default | |
config: | |
user.vendor-data: | | |
#cloud-config | |
package_upgrade: true | |
packages: | |
- build-essential | |
- software-properties-common | |
users: | |
- name: ubuntu | |
ssh-import-id: gh:jamestomasino | |
shell: /bin/bash | |
description: Default LXD profile | |
devices: | |
eth0: | |
name: eth0 | |
nictype: bridged | |
parent: lxdbr0 | |
type: nic | |
root: | |
path: / | |
pool: zfs | |
type: disk | |
name: default | |
used_by: [] | |
The key here is the ssh-import-id to pull my keys down from | |
github. With that I can ssh in immediately instead of using the | |
lxc exec method to sudo in. | |
Step 5: Edit the gui profile | |
If you didn't install lxd via snap, you'll need to run this one liner: | |
$ echo "root:$UID:1" | sudo tee -a /etc/subuid /etc/subgid | |
config: | |
environment.DISPLAY: :0 | |
raw.idmap: both 1000 1000 | |
user.user-data: | | |
#cloud-config | |
runcmd: | |
- 'sed -i "s/; enable-shm = yes/enable-shm = no/g" /etc/pulse/client.co… | |
- 'echo export PULSE_SERVER=unix:/tmp/.pulse-native | tee --append /hom… | |
packages: | |
- x11-apps | |
- mesa-utils | |
- pulseaudio | |
description: GUI LXD profile | |
devices: | |
PASocket: | |
path: /tmp/.pulse-native | |
source: /run/user/1000/pulse/native | |
type: disk | |
X0: | |
path: /tmp/.X11-unix/X0 | |
source: /tmp/.X11-unix/X0 | |
type: disk | |
mygpu: | |
type: gpu | |
name: gui | |
used_by: [] | |
This magic sauce will set up all the dependencies needed to | |
connect to my display when a gui app is run. I then SSH in with | |
export x11 enabled and all is well. | |
Step 6: Edit a user profile | |
This one involves a lot of runcmd: stuff in a row that points to | |
my dotfiles, installs other apt packages, does a little dance, and | |
ends up with a working environment for me to do some damage. I'll | |
spare you all. | |
Step 7: SSH | |
I want to be able to connect to lxd containers by name, not just | |
IP address. I'm going to set that bit up in a minute using dnsmasq | |
and systemd, but first lets look at my ~/.ssh/config to see what | |
goodness lies within: | |
Host *.lxd | |
StrictHostKeyChecking no | |
UserKnownHostsFile /dev/null | |
LogLevel QUIET | |
ForwardX11 yes | |
ForwardX11Trusted yes | |
User ubuntu | |
IdentityFile ~/some/really/cool/path/to/SECRETS | |
The host checking and host file bit, along with LogLevel mean you | |
won't get man-in-the-middle warnings if your container IP changes | |
one day from starting and stoping repeatedly. X11 forwarding there | |
for display. I use ubuntu containers, so I've attached the | |
username there as well. | |
Step 8: DNS | |
$ sudo vim /usr/local/bin/lxdhostdns_start.sh | |
#!/bin/sh | |
LXDINTERFACE=lxdbr0 | |
LXDDOMAIN=lxd | |
LXDDNSIP=$(ip addr show lxdbr0 | grep -Po 'inet \K[\d.]+') | |
/usr/bin/systemd-resolve --interface ${LXDINTERFACE} \ | |
--set-dns "${LXDDNSIP}" \ | |
--set-domain ${LXDDOMAIN} | |
$ sudo vim /usr/local/bin/lxdhostdns_stop.sh | |
#!/bin/sh | |
LXDINTERFACE=lxdbr0 | |
/usr/bin/systemd-resolve --interface ${LXDINTERFACE} --revert | |
$ sudo vim /lib/systemd/system/lxd-host-dns.service | |
[Unit] | |
Description=LXD host DNS service | |
After=multi-user.target | |
[Service] | |
Type=simple | |
ExecStart=/usr/local/bin/lxdhostdns_start.sh | |
RemainAfterExit=true | |
ExecStop=/usr/local/bin/lxdhostdns_stop.sh | |
StandardOutput=journal | |
[Install] | |
WantedBy=multi-user.target | |
$ sudo systemctl daemon-reload | |
$ sudo systemctl enable lxd-host-dns.service | |
$ sudo systemctl start lxd-host-dns.service | |
Boom. If you didn't answer defaults on lxd init, then your | |
nterface might not be lxdbr0 and you'll need to change stuff. | |
Step 9: Test it out | |
$ lxc launch ubuntu: test | |
$ lxc list # wait until you can see the IP to know it's ready | |
$ ssh test.lxd | |
If everything blew up it's because you're with me and that DNS | |
hackery works great to be able to ssh in, but it also breaks the | |
ability for your container to connect to anything else. For now | |
I'm disabling the DNS bit and sshing in via IP, or using the lxc | |
exec sudo stuff to connect. I'm watching this post [0] for | |
a comment response that will hopefully clarify that last bit. | |
[0] How to use lxd container hostnames on the host in Ubuntu 18.04 |