Subj : Re: Monitoring BBS logs
To   : Mike Miller
From : Dan Clough
Date : Thu Feb 01 2024 08:04 pm

-=> Mike Miller wrote to Dan Clough <=-

DC> Hello all,

DC> Wondering what you all do/use for monitoring logs on your boards.

DC> I have a few terminals open to "tail" various logs on my system
DC> (running Synchronet).  Been doing it that way for a long while, and it
DC> works fine, I guess.

DC> Has anyone used the 'lnav' utility to watch logs?  It's pretty slick,
DC> has LOTS of options, and I like the use of colors in logs to help
DC> things "jump out" at you.  If anyone is using this, and has taken the
DC> time to customize the look/feel of how it displays, would you comment
DC> on how you're using it, and would you maybe be willing to share your
DC> custom config that I could use as a starting point to learn how it's
DC> done?

DC> Appreciate any input you may have!


MM> The free grafana-cloud tier is plenty for monitoring my bbs
MM> server.  I'm streaming the logs to it via the grafana-agent.

MM> Last week I got a wild hair and set up this:

MM> https://snapshots.raintank.io/dashboard/snapshot/ybCmnOlQAJnFye872
MM> Dsmdi1ev7EV038q

MM> (snapshot from the last 24 hours as of this post)

Wow, that's pretty awesome!

MM> I still plan on adding number of messages from/to
MM> uplinks/downlinks.  I may actually send out alerts if dupes rise
MM> above a certain pecentage.

MM> I do have another dashboard that I'm working on. It has number of
MM> connections over time, number of connections by IP address, and a
MM> few other things.

MM> doing this via logs is not all that efficient, though. At some
MM> point I'll likely create some kind of prometheus exporter to just
MM> scrape data, rather than forcing grafana to parse its entire
MM> history of logs (only 48 hours worth on the free tier) to display
MM> data.

I have no knowledge of this kind of thing, seems above my comfort level
for sure.  The free tier does seem to be fairly useful, if one can grasp
the overall big-picture of what it's doing.  I may create an account and
try to figure some of it out.  Thanks for pointing this out.

MM> Grafana will, however, allow you to set up alerts based on
MM> triggers,  such as a specific line in a log file.  SBBS choked on
MM> me in a weird way the other day, for the first time in forever,
MM> so I may set it up to send an alert to my phone if something like
MM> that happens again.  I didn't catch it, and the board was down
MM> for more than a day.

That sounds like a great feature too.  Thanks again for your reply and
this information.

Dan



... System halted - Press all keys at once to continue.
=== MultiMail/Linux v0.52
--- SBBSecho 3.20-Linux
* Origin: Palantir * palantirbbs.ddns.net * Pensacola, FL * (1:135/115)