Quoting myself from [1]:
Dear UNIX diary,
today, I exploited memory overcommit and copy-on-write.
I have a program that reads data and stores it in a circular buffer.
That is, when the buffer is "full", new data is simply stored at the
beginning of the buffer, thus overwriting old data. The program just
keeps the data in ordinary memory, i.e. it does not save it on a hard
drive or something else.
This program can be fed with arbitrary data, for example raw audio
data. This means you keep recording and recording, but you will
always only have ~50MB (or the last ten minutes) of audio data
available.
So far, so good.
What I want to do next is send my program a signal and it shall save
the buffer as an OGG/Vorbis file. Meanwhile, I still want to keep
recording to the very same buffer -- without any interruption. Of
course, as new data arrives, old data will be overwritten, so I must
make a copy of my buffer prior to encoding it. Creating that copy
could be a problem if the original buffer is, say, 80% of my total
RAM.
Enter: Kernel magic.
Okay, I have a process with some allocated memory. I can solve my
problem by simply doing a fork(). The child process will be a clone of
the parent -- including its memory. The child can then encode the
audio data as OGG/Vorbis while the parent continues to record new
data.
What's awesome about this is that the kernel does not really create a
copy of the memory[1]. Both processes refer to the same memory pages.
Only when one process writes to a "shared" page (the parent does this
when recording new audio data), a real copy will be created. Thus,
only that particular data in my circular buffer which arrives while
I'm encoding needs to be copied. This only amounts to a fraction of
the total buffer size, which allows me to do the whole operation even
with very large buffers.
And, I, myself, don't have to implement any of this. The kernel takes
care of everything.
[1]: At least, Linux behaves this way by default. Last time I checked,
FreeBSD and OpenBSD did the same thing.
____________________
Oh, and yes, I finally did some UNIX programming in C again. Has been
a while. The life of a sysadmin can be tedious at times.
____________________
Another internet outage today. I was online using my cell phone (over
bluetooth) for a couple of minutes.
This is amazing. Everybody should browse the modern web like this
every once in a while. Most importantly, web designers should do this.
I got 25 kB/s and I could barely do anything. Almost impossible. When
you open a modern web site, huge amounts of data needs to be
transferred. I remember browsing the web using ISDN which was able to
provide 7 kB/s and that was just fine. But times have changed. In
Germany, many people have internet connections which are literally a
thousand times faster than ISDN. Of course, nobody cares about
bandwidth anymore.
I wonder what it's like to browse the web from some other countries.
____________________
1.
https://nixers.net/showthread.php?tid=1539&page=13