* * * * *

          “Multithreaded programming is hard, let's play Minecraft.”

All I wanted to do was pass some data from one thread to another. I had code
to manage a queue (based off the AmigaOS list code [1]), so that wasn't an
issue. And while a pthread mutex [2] would ensure exclusive use of the queue,
I still had an issue with one thread signaling the other that data was ready.

I took one look at pthread condition variables [3], didn't understand a word
of it, and decided upon a different approach.

The primary issue really wasn't the transferring of data; it was the
processing to be done with the data. I'm writing code to drive, let's say, a
widget, for “Project: Wolowizard [4]” and part of that test code is
simulating, say, a sprocket. When the sprocket gets a request, the original
code could return a result or just drop the request. R wanted a way to delay
the results, not just drop them. To avoid a redesign of the test program, the
easiest solution was to just spawn a separate thread to handle the delayed
replies.

So, not only did I have to transfer the request, but signal the other thread
that it had a new request to queue up for an X second delay.

My first test approach (a “proof-of-concept”) used a local Unix socket for
the transfer. This approach had the benefits of avoiding the use of mutexes
and condition variables, and the code was fairly easy to write. Just poll()
the local Unix socket for data with a calculated timeout; if you get a
timeout, then it's time to send the next delayed result, otherwise it's a new
request to queue (ordered by time of delivery) and recaluclate the timeout.

But I found it annoying that the data was copied into and out of kernel space
(not that performance is an issue in this particular case; it's more of an
annoyance at the extra work involved).

Fortunately, I found sem_timedwait() and sem_post(). I could use
sem_timedwait() to wait for requests, much like I used poll(). Some mutexes
around the queue (I originally used some other semaphores, but M suggested I
stick with mutexes for this) and the code was good to go.

Only it didn't work in the actual program. My “proof-of-concept” worked, but
in actual production, the second thread was never notified.

I asked M for help, and through his asking the right questions I suddenly had
an idea of what was wrong—one quick change to a test script (the testing
program I wrote is driven by Lua [5]) proved my hunch right. Which is when I
planted my face in the palm of my hands.

The upshot—my sprocket simulation works in one of two different ways (one
method uses The Protocol Stack From Hell™ as a test) and the way I picked
(which doesn't use The Protocol Stack From Hell™ and is thus, more “safe,” as
both M and I are of the opinion that The Protocol Stack From Hell™ is also
subtly buggy) failed to trigger the appropriate semaphore. Worse, had I used
the local Unix socket, the whole thing would have worked as intended.

One of these days I'll get this whole multithreaded programming thing [6]
down pat.

[1] http://amigadev.elowar.com/read/ADCD_2.1/Libraries_Manual_guide/node02D8.htm
[2] https://computing.llnl.gov/tutorials/pthreads/#Mutexes
[3] https://computing.llnl.gov/tutorials/pthreads/#ConditionVariables
[4] gopher://gopher.conman.org/0Phlog:2010/10/11.1
[5] http://www.lua.org/
[6] gopher://gopher.conman.org/0Phlog:2007/10/22.2

Email author at [email protected]