Subj : binkd crashes when reloading after file change detection
To   : Dan Cross
From : mark lewis
Date : Fri Jan 21 2022 10:40 am


On 2022 Jan 21 12:57:30, you wrote to me:

DC> These sort of annoying "heisenbugs" are the worst.

you can say that again! :lol:

DC> But fortunately there are tools that can help out.  Perhaps compile
DC> with something like TSAN enabled and try running that way?

i have thought about attempting to build a debug version of binkd and running the core file through gdb to get a backtrace when the fecal matter gets flung about... unfortunately my basic knowledge and use of the make system gets in the way of doing that... i've also thought about doing the build and running it with valgrind... but again, my basic knowledge of make gets in the way...

i don't have a problem providing backtraces to the devs as long as getting them is easy to do and doesn't require much in the way of gymnastics... i do this now with several other projects... most of them use cmake which i also know next to nothing about...


[edit]

FWIW: it looks like there is some sort of memory consumption in the client section of the code... whether it is a leak or not, i don't really know... the client side is the one that has crashed on me when reloading the configs in the past...

the following is from my monitoring script i whipped up yesterday... the first reading is immediately after loading binkd... the bottom one is after binkd has been running for just over 24 hours servicing some 40 or 50 links and whatever random connections came in... it stays quite busy transferring mail and files... in the config i have set for 1 server and 1 client which makes it easier on me to keep track of sessions in the log...

********** Program start at 2022-01-20 10:22:39 -0500 **********
Current time: 2022-01-20 10:22:39 -0500
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
sbbs     23128  1.0  0.3  13980  3104 pts/1    S+   10:22   0:00 binkd: server manager (listen 24554)
sbbs     23129  0.0  0.2  14028  2412 pts/1    S+   10:22   0:00 binkd: client manager

[...]

Current time: 2022-01-21 10:27:46 -0500
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
sbbs     23128  0.0  0.1  13980  1696 pts/1    S+   Jan20   0:01 binkd: server manager (listen 24554)
sbbs     23129  0.1  1.1  25496 11752 pts/1    S+   Jan20   2:44 binkd: client manager


from another terminal, i manually triggered a compile of the nodelist that arrived just after local midnight and Boom! we triggered the crash in the client manager again... the server manager didn't even get a chance to notice the change and update its config this time...

+ 10:30 [23129] /sbbs/ftn/nodelist/fidonet.binkd.txt changed!
+ 10:30 [23129] Reloading configuration...
 10:30 [23129] previous config is no longer in use, unloading
free(): double free detected in tcache 2
! 10:30 [23128] client manager (pid=23129) exited by signal 6

this seems to add more credence to the problem being somehow related to long running times and possibly some sort of memory leak or corruption...

)\/(ark

"The soul of a small kitten in the body of a mighty dragon. Look on my majesty, ye mighty, and despair! Or bring me catnip. Your choice. Oooh, a shiny thing!"
... I multitask... I read in the bathroom.
---
* Origin:  (1:3634/12.73)