OpenBSD has this policy called "secure by default":
https://www.openbsd.org/security.html
The gist of it is that you have to explicitly enable things that might
be "dangerous", like daemons reachable from the network. So, on a
freshly installed system, there are no surprises (at least not in my
experience so far).
On Arch Linux, the situation used to be somewhat similar in practice,
although I don't think that "secure by default" is actually a goal of
the project. It's more of a side effect coming from the distro's
"simplicity". The system was clean after initial installation and
installing new packages did not enable new daemons. The latter has
slowly changed since the introduction of systemd about 10 years ago
(enabling daemons is now easy for the package manager to do), so you
have to be more careful now.
I really like this "secure by default" approach. Yes, it's more work for
me and less "comfortable", because things sometimes don't "just work",
but I like to be in control of my OS and I really don't like surprises
in the realm of security.
On the other end of the spectrum, you have systems like Kubernetes. (I'm
not comparing OpenBSD and k8s, they're completely different tools -- I'm
comparing the mindsets.)
I recently found out about this:
https://kubernetes.io/docs/concepts/storage/volumes/#hostpath
In a regular deployment manifest, you can -- by default -- mount paths
from *the host*. This was completely unexpected to me. What this means
in practice is that anyone who can deploy stuff into a cluster is
effectively root on all k8s host nodes.
What you can do with this aside from some obvious things: You can mount
`/var/run/docker.sock` as a `hostPath` in your k8s container. You now
have access to the Docker daemon running on your *host* and you can run
whatever you want, without k8s knowing about it.
You might respond with: "Well, then don't let people deploy random
things, duh?!" Yes, maybe you're right. Let me elaborate, though.
First of all, `hostPath` is a rarely needed feature for normal
operation. You'd usually never do this; if anything, you'd mount some
managed persistent volume, but not a path on the host node. And the
official docs even have a big red security warning. This is meant to be
used only when you really know what you're doing, and still, this
feature is enabled by default. I now have to research how to *disable*
it. That's just the wrong way around, if you ask me. It also makes me
wonder how many other similar loopholes exist. How am I supposed to find
them all? Kubernetes is a behemoth. Read through the *entire* k8s docs
(and do so after every upgrade)? Read guides like "how to harden your
k8s cluster"? Again, I think this is backwards -- it just puts more
burden on admins.
Second, Docker and k8s often create the impression that they're about
"isolation". Sometimes they are even marketed as such. I admit that I
believed this for quite some time, too. I slowly learned over time that
it's not true, but `hostPath` still left me baffled. Here's what we
planned to do at work (we eventually didn't implement this, luckily,
only a few people use k8s now): Have the admins set up a k8s cluster and
grant *all devs* access, so they can develop and deploy their programs
in the cluster. This would have made the lifes of our devs much easier,
because they wouldn't have to contact the admins anymore when they
started a new project. This is the dream that many k8s fans have: No
more admin interaction, we can just deploy our stuff, everything is
neatly isolated.
Now, do I trust our devs? Yeah, kind of, but not really. They're not
malevolent, don't get me wrong. However, devs think differently than
admins, in my experience. Here's how I found out about `hostPath` in the
first place: Our devs created some Jenkins jobs (which run in k8s in
build pods) which build new Docker images. To do that, you need ... a
Docker daemon. Do you have a Docker daemon inside a k8s pod? No. So,
what did they do? They mounted `/var/run/docker.sock` from the k8s host
and ran their builds outside of k8s. (Those are long-running, heavy
tasks.)
This is something that no admin would ever do, I'd argue. This is a
"hack", a "dirty workaround". That host is meant to be managed entirely
by k8s, it shouldn't run random Docker images. How does that impact k8s
operation (scheduling, shutdown, volumes, image caches, does that still
work after the next k8s upgrade, ...)? I don't know, hence I wouldn't do
it.
Our devs did it, though, because they have a *different job*. I don't
blame them in the slightest. It's not their job to think about these
issues. They had a problem, found a solution, it worked, okay, done.
What I do blame is Kubernetes. I had hoped and assumed that things like
that simply don't work -- unless, maybe, it was explicitly allowed. But
it's *the default*.
I don't think I'll research how to disallow usage of `hostPath` (too few
people actually use k8s, it's not worth it). This feature has undermined
any kind of trust I had left for k8s, though.
