* * * * *
The point of writing software seems to be writing tests for the software, not
in running the software
I've been stuck in testing hell the past few weeks, what with the changing
[1] of the [2] old guard [3], and the new [DELETED-guards-DELETED] crew are
heavy into the whole “unit test über alles” religion. I understand the reason
behind testing, and unit testing in particular. The few times I've tried unit
testing, it has been helpful, but …
1. I think it works best when you start out with unit testing in mind;
2. it works best with a module or library;
3. you test at the boundary of the module or library and not the
implementation;
4. you do have tests for your tests, right?
That last bit might sound a bit glib, but it is a real concern. Even Knuth is
quoted as saying, “beware of bugs in the above code; I have only proved it
correct, not tried it.” I even joked about having to test our tests with both
my current (but soon-to-retire) manager and his (soon-to-me-my) manager,
because our current regression test has 15,852 individual tests (not all of
them might be valid—hard to say, since the tests are automatically
generated).
“Project: Lumbergh [4]” was not written with unit tests in mind. Or more
specifically, it is a unit. The whole thing. And it's complicated because not
only does it implement our business logic (which over ten years has gotten
quite complex) but because it has to query multiple databases at the same
time across the network (“at the same time” because it has to do its job as a
phone call is being made). How does one deterministically test delayed
(and/or dropped) queries?
Another issue—“Project: Lumbergh” sends a message to “Project: Cleese [5]” in
some circumstances (“Project: Cleese” handles an HTTP (HyperText Transport
Protocol) notification on behalf of “Project: Lumbergh” because trying to add
a TCP (Transmission Control Protocol) connection in a program that deals with
UDP (User Datagram Protocol) would have taken too much engineering time) and
we need to check if “Project: Cleese” was notified. I solved that one by
mocking the HTTP endpoint that “Project: Cleese” talks to with two
interfaces, one for “Project: Cleese” and one for the regression test to
query. The issue there is the regression test might ask the mock if it
received a request from “Project: Cleese” before “Project: Cleese” gets the
notification from “Project: Lumbergh” (a classic race condition). I got it
working reliably, but now the regression tests takes over twenty minutes to
run, instead of the two-plus minutes (even on the happy path, and I'm not
entirely sure why).
It seems like management is more concerned about the tests rather than the
product.
[1]
gopher://gopher.conman.org/0Phlog:2020/10/28.3
[2]
gopher://gopher.conman.org/0Phlog:2020/10/29.1
[3]
gopher://gopher.conman.org/0Phlog:2020/11/23.1
[4]
gopher://gopher.conman.org/0Phlog:2018/09/11.2
[5]
gopher://gopher.conman.org/0Phlog:2018/09/11.2
Email author at
[email protected]
