ROOPHLOCH 2023 explanation post
-------------------------------
Well, it's the last day of the month already! I guess strictly
speaking it's perfectly valid to make a ROOPHLOCH post itself during
September and then explain exactly how you did it in a follow-up
post in October, but I don't want to leave this too long. I made
my post[1] back on the 19th, in case you missed it.
Way back when I lived in Auckland, I bought a pair of of 433 MHz
radio modules made by the Chinese company Dorji, one transmitter
and one receiver, from the (now sadly defunct!) Surplustronics
store on Queen Street. These operate in a "Industrial, Scientific
and Medical device" (ISM) frequency band which is available for
license free use pretty well everywhere on Earth and is heavily
used in basic wireless consumer devices like garage door openers,
doorbells, hobby weather stations and stuff like that. I was
pretty heavily into 8-bit homebrew computing at the time and was
still extremely naive about anything to do with radio. I don't
think I was even listening to shortwave broadcasts at this point
in time. I vaguely recall that I bought these modules hoping they
would let me easily add remote peripherals to my Z80 machine.
They have a super basic interface, the transmitter has a single
digital input pin and the receiver has a chip select pin (to switch
the receiver in and out of a low power standby mode) and a single
digital output, and aside from power connections that is all of it.
I think I hoped I could just hook these up between UART or SPI/I2C
devices and it would "just work", basically carrying an arbitrary
5V TTL signal from point A to point B.
It didn't "just work". These are Amplitude Shift Keying (ASK)
devices. When the transmitter is turned on, it emits a 433 MHz
carrier continuously at one of two different non-zero amplitudes,
one corresponding to a logic low input and the other to a logic high.
If you think about this a moment, it's clear that when you hold the
input steady, the RF output is entirely ambiguous to a receiver.
The receiver can tell there's a carrier there with some signal
strength, but is that a low power carrier from a nearby device or
a high power carrier from a distant device? The whole thing only
works if the data signal is transitioning between the two states
frequently enough that the receiver can "lock on" by holding the
RF amplifier gain at the right level relative to the decoding
comparator threshold. Proper devices using this system rely
on things like Manchester encoding schemes to ensure that the
transmitted bitstream flips between 0 and 1 constantly, even if
the input bitstream includes long constant runs. Even trickier,
the receiver has no squelch functionality. If the receiver is off
and there's no carrier floating around, the transmitter just cranks
the gain way up until random noise induces chaotic high speed bit
flipping in the output. So proper devices again usually precede
any actual transmissions with a kind of "wake up" sequence where
a known bitstream is repeated over and over again enough times
that the probability of that exact sequence appearing by chance is
effectively zero, and the receiver has to constantly poll for it.
This is all very workable but it's waaay more complicated than what
I was looking for at the time, which was basically a magic invisible
wire to carry logic level signals from point A to point B, so these
guy have sat idle in my junk box ever since.
They've been in the back of my mind every time ROOPHLOCH has rolled
around, but this year I finally decided to try to put them to
good use. Last year, when I kind of dropped off the internet for
nine months or so, I got pretty heavily into homebrew radio projects
and spent a lot of time and energy in particular on decoding digital
maritime utility signals, especially NAVTEX on 518 kHz, but also some
HF NBDP stations and later HF DSC. This is still a subject pretty
close to my heart and it's kind of astonishing that I have written
*zero* words about it on Gopher/Gemini. It'll happen at some point,
I promise you all! Anyway, these are all Binary Frequency Shift
Keying (BFSK) signals, where the RF carrier switches between two
slightly offset frequencies to represents 0s and 1s, at 100 baud in
all these applications. It's exactly the RF equivalent of the AFSK
schemes used to represent bit streams in audio by both early modems
and cassette-based storage systems in home computers, and in fact
the standard way to decode these signals, as a hobbyist at least,
and I guess in the pre-SDR era that I'm still heavily invested in,
is to shift the signal down to the audio range with an SSB receiver
and then feed that into a soundcard. All of which just means that,
unlike my radio-naive earlier self when I bought these modules,
this year the question of "how do I easily transmit binary data
using these things that only work with constantly alternating input
streams and receive random noise in the absence of such?" had a
blindingly obvious answer.
I didn't want to build anything permanent for the sake of this
ROOPHLOCH experiment, both to save time soldering and avoid
committing parts to what might not work very well, and I also
didn't want to invest very much effort in programming. So for the
first time in a long time, I pulled out my Arduino. Or, strictly
speaking, I grabbed my "Eleven", a 100% Arduino Uno-compatible
AVR dev board made by the Australian company Freetronics, which
I purchased from a JayCar circa 2010 when I was just starting to
get back into electronics again. It was my first encounter with
microcontrollers, and I used it enthusiastically for a year or two
before I learned how to use AVR microcontrollers directly, using
avr-gcc and avrdude in conjunction with a USBtinyISP programmer.
It's been pretty unloved since then, but it comes in handy for
very quick-and-dirty proof-of-concept projects like this once.
I set one of the Arduino's analog outputs to 128, i.e. the midpoint
if the 8-bit output range. Because the Arduino doesn't have a true
DAC but uses PWM, this caused a 50% duty-cycle square wave to appear
on an output pin. With the default settings, the PWM frequency is
a surprisingly low ~1 kHz, very much in the audio frequency range.
I fed this into the input of a 4040 divider chip, which gave me
an additional 500 Hz squarewave output (also 250 Hz, 125 Hz, etc.,
but I made no use of these). The 1 kHz and 500 Hz waves were fed
into two of the inputs of a 4052 analog multiplexer, the output of
which I fed into the 433 MHz transmitter. This basically meant
that by toggling a single one of the 4052's input selector pins,
I could feed the transmitter with a squarewave at one of two audio
frequencies. Because a squarewave is constantly alternating,
it travels very cleanly through the ASK modulation/demodulation
chain and basically this setup just beams audio frequency square
waves from one circuit to another. By toggling a single pin you
shift the pitch of the audio tone, and you can toggle the tone as
slowly as you like or hold it steady for arbitrarily long periods.
At this point, you could user various hardware FSK decoders like
a pair of LM567s with some glue or an NJM2211 to recover logic
high/low outputs and then you really could use these modules to
pass the signal from a UART or SPI/I2C bus without worrying about
the fact that they idle at a fixed level (in this scenario I'd
probably use a quartz oscillator instead of an mcu PWM output).
However, for ROOPHLOCH I wanted to keep things fast and simple,
and since the two frequencies were in the audio range and I had
been dreaming of using minimodem since the very first ROOPHLOCH,
I decided to just treat the receiver output as audio. After all,
the message had to get into a computer anyway in order for me to
be able to push it into Gopherspace.
My initial plan was to use the Arduino's UART output to drive the
multiplexer, so that the firmware "sketch" could be as simple as
Serial.println()-ing my post. This didn't actually work out at
all, and a little consideration of timescales makes it clear why.
At the default baudrate of 9600, the UART switches state once every
104 microseconds, while the two squarewaves have periods of 1000
and 2000 microseconds, so the tones don't come through cleanly
at all. Without changing the Atmega328's clock frequency (which
would presumably stop the Arduino IDE being able to talk to the
bootloader), its UART baudrate generator can't go any lower than
about 240 baud. I probably could have played around the the timer
prescalers and bumped the audio frequencies up an octave or two and
found a baud rate that worked, but in the end I took the lazy way
out and just resorted to bit-banging UART output on a digital pin.
At a relaxed 100 baud, you can do it using delay() like a noob
rather than bothering with interrupts and it still works.
At first I fed the 5V squarewave output from the receiver through
a resistor divider and then into my stereo, planning to use the
microphone in my Thinkpad to receive the audio, but the decoding from
this was surprisingly noisy. So I fed the divided output directly
into the input RCAs of my trusty Behringer UCA202 instead and it
became absolutely bullet-proof. I could burn the text of a post into
the Sketch (by keeping the post short it fit within the 2 kB of RAM
and I didn't need to use PROGMEM techniques) and then recover that
text completely using minimodem. I was pretty pleased at this point!
All that remained was to automate the process of receiving
and publishing the post, so I could set a script running on my
laptop indoors, then take the Arduino outdoors, plug it into a
USB powerbank and wait, using my phone to check Gopher and see if
the post had appeared. I spent a while experimenting with various
minimodem options to get this as solid as I could. The --rx-one
option will cause minimodem to exit after a single gain and loss
of carrier event, and --quiet will stop it reporting those events
and various other diagnostics. Using the --confidence option to
set a high confidence threshold makes sure that minimodem is very
unlikely to misinterpret a burst of noise as a very quick gain and
loss of carrier, which in conjunction with --rx-one would cause
the script to exit while I was still on my way down the stairs.
Long periods of indoor testing confirmed that with this suite of
options, I could start the script running and leave it running for
even an hour with the transmitter turned off without any false starts
- I had a very stable "standby" mode. When powering up the Arduino,
the text of the post would usually be received reliably but there
were two failure modes which happened often enough to worry me.
One was simply that the first one or two characters were dropped,
but minimodem would also sometimes terminate after receiving a single
garbage character. I adjusted the sketch to idle with the UART
pin high for ten seconds before it started actually transmitting,
and I also had it transmit 10 0x02 bytes before starting to send
content, so I could leverage minimodem's --sync-byte option, which
will suppress carrier acquisition until it has received a certain
byte multiple times. 0x02 is the ASCII control code for "Start of
Text", so it seemed appropriate to use here. I used `tee` to capture
the output of minimodem run with all these options into a .txt file,
had the script use `wc` to double check that more than a single line
had been received (to protect from publishing one of those mysterious
single garbage characters), and then simply did a git add, commit
and push to publish. I manually updated my Gophermap to include
a link to the received file before starting the script running.
I positioned my laptop with the receive script running near a window,
bundled the Arduino and the breadboard with the 4000 chips and the
433 MHz transmitter into a shoebox, with a USB cable to power the
Arduino dangling out from under the lid, put a USB powerbank into
my pocket and headed outside. This was by far my most technically
ambitious ROOPHLOCH post yet, but also by far the least ambitious
in terms of how "remote" a location I was posting from. Before I
even got as far as doing anything with minimodem, I did a few rough
and ready range tests where I just fed the transmitter a square
wave, displayed the receiver's output on an oscilloscope screen,
took the transmitter outside, phoned my wife and asked her to tell
me when the scope stopped showing a nice clean, steady square wave
and started flickering with transient garbage, while I wandered in
various directions to figure out the plausible boundary of where I
could post from. Reception seemed pretty solid out to 50 meters,
but I didn't want to take any risks for the actual post, so I
just sat on the doorstep of a closed cafe on the other side of the
street from me and plugged the power bank in and hoped for the best.
A couple of Mormon missionaries strolled along while I was waiting
and I was briefly afraid they would take advantage of the unusual
scene to ask me why I was trying recharge a shoebox before as an
icebreaker before trying to convert me, but in the end they just
passed me by. I kept furiously refreshing PocketGopher waiting for
my post to appear, cursing myself for being too lazy to put an LED on
the breadboard to signal when transmission was complete. Eventually,
dejectedly, I admitted to myself that it obviously hadn't worked.
I came back upstairs and confirmed that minimodem had had one of its
spurious receive events, but my `wc` sanity check had spared me the
embarrassment of posting it. I decided to give it one more shot,
started the script up and took my box outside again. Sat in the same
location, did exactly the same thing. And it worked!!! It worked
perfectly, with no dropped characters, you'd never know that it was
uploaded in such an unusual way. I aspirationally titled the post
"transmission 01", as I hoped to try a few additional experiments,
including a solar powered post. In fact, I *did* try the solar
post, with low expectations as I was using a very dinky little
5x10cm panel and the linear regulator on the Arduino board is not
exactly highly efficient. To my surprise, the LEDs on the Arduino
stayed on and solid and bright under midday sun, but I minimodem
never picked up a thing, and I didn't make the time to dig into it
too deep. It's nice to leave some targets for next year, anyway.
I also wanted to try to increase the range of the setup, but
didn't end up making time for that either. I did make a little
coil-loaded antenna for the receiver using some solid copper
wire, but it didn't actually seem to help any. I didn't add an
antenna to the transmitter. ISM devices are required to accept
any interference they encounter because it's a shared band and
nobody is guaranteed exclusive access, but I wanted to try to be a
good neighbour during all my experimenting and not stomp on other
people's signals, so trying to optimise the receiver and leave the
transmitter modest seemed appropriate. It seemed to work pretty
solidly out to 50 meters. Maybe I could have mounted the receiver
outside a window to get some extra distance without too much effort.
Folks online have built directional Yagi antennas for modules like
this and reported good results. I'm pretty sure 100 meters would
be achievable without extreme efforts, and that would be enough
that I could at least sit in a grassy area with some trees behind
my local supermarket. I mean, it's not exactly a salubrious locale,
but at least it has a vaguely outdoors vibe. Something else to try
next year, perhaps. It's slightly ridiculous in this day and age
of LoRaWAN (which also functions, at least in some parts of the
world, in the 433 MHz ISM band) to continue trying to coax what
is basically a garage door opener into a reliable long distance
communication tool, but this kind of thing is exactly my jam.
Where's the fun in buying stuff that just works out of the box?
[1]
gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/phlog/roophloch-2023-433-mhz-transmission-01.txt
