System design and "Correctness"

I get the feeling there are lots of cases where software asks the OS to do something like "flush this to disk" or "deallocate this memory", and systems like OpenBSD say "sure, I'll do it, and I'll do it correctly" and Linux says "you can't possibly mean that, so I'll just put your request into a queue and process it the way I think will work more smoothly," so Linux ends up running faster and more smoothly and developers rely on certain things not actually mattering because they won't actually be taken literally by the OS anyway.

Which ... well, kind of makes sense, since the underlying hardware increasingly takes instructions as a suggestion anyway now.

Which unfortunately means that OpenBSD doing things precisely like that is exactly why a good number of 3rd party software runs like crap on it? right?

Though I think ultimately if like 90% of the devs of any cross platform software run Linux and less than 1% run OpenBSD, that's probably your best predictor of Linux performance being more likely to be better than OpenBSD performance for said software.

As I understand it, could be wrong, syscalls request the kernel to do something to help you do something in userland. So something like chimera using musl and a \*BSD hybrid userland requesting something from the Linux kernel wouldn't look too much different than Debian with glibc and gnu userland requesting something from the linux kernel internally.
Its all mostly POSIX under the hood regardless of \*BSD or Linux or whatever unixen, its just the kernel implementation details not defined in POSIX that i'm talking about here.

There is a syscall, fsync() which flushes disk caches:

- When you call fsync() on OpenBSD, the system takes it at face value: "flush this to disk" means exactly that. It'll go through the kernel buffers, push through any write caches it can reach, and try to make sure the data actually hits the disk. No shortcuts, no cleverness. It's predictable, and if something goes wrong, you at least know why- because the kernel doesn’t lie to you.
- Linux handles fsync() a bit differently. It's still doing what it's supposed to do, but it also tries to optimize for performance. Depending on the filesystem (like ext4 or xfs), the kernel might delay some actual disk writes or coalesce them together for efficiency. And if your drive has a write-back cache, you might need extra flags or mount options to make sure the flush really happens. So yeah, it’s technically compliant, but there’s some extra "gotchas" there.


Same deal with free():

- OpenBSD is aggressive about giving memory back to the system. If a chunk was mmap'd, it'll likely munmap it as soon as you free it. Even the malloc() implementation is set up to release memory when it makes sense, and everything gets scrubbed before reuse to prevent leaking old data. It's slower, but also cleaner and safer.
- Linux, on the other hand, keeps freed memory around in userspace for reuse. glibc (or whatever libc for that matter) doesn't usually hand it back to the kernel unless it's a big mmap'd block. The idea is that you might need it again soon, so why go through the overhead of unmapping and remapping? From a performance standpoint, that’s smart. The tradeoff is you’ve got to trust the allocator not to leak anything to the rest of the userland...and you don’t get that same "warm fuzzy feeling" that whatever you malloc'd is gone that you get on OpenBSD.

I honestly would rather the underlying os be implemented in a way that ensures functional correctness as a proactive measure in stability, even if it means a hit in performance. But even then OpenBSD performs pretty alright for me. It's no speed demon tho.

tags: system-design, correctness, unix, linux, openbsd, syscalls