Sunday, August 6th, 2017

       Bongusta! and how it works
       ~~~~~~~~~~~~~~~~~~~~~~~~~~

So I created this phlog aggregator called Bongusta![1] and people seem
to like it and even use it. I was  already asked for the  source code,
but as I'm not  exactly good in sharing my  sources, I'll try at least
to  describe, how  Bongusta! works. I may later upload the source code
to my  GitHub  repository [2], but let me polish it a bit before that.
Anyway:

The  whole  aggregator is just a  single Perl  script,  invoked  every
midnight  (CET/CEST  time) by  the  cron  daemon.  There is a  list of
aggregated  phlogs in the  form of  name => URL  hash. Not much to say
about     the  " name"  part of the  hash,  it's  just the name of the
particular phlog and is  displayed  exactly as is. But as some  phlogs
don't have a single URL for all posts, but more  complicated structure
like  subfolders by  month[3], the URL has to be a bit more  flexible.
For such phlogs the URL can contain two special strings: #yyyy# and/or
#mm#.

In the first step the  script goes  through the hash and  replaces all
special  strings with value  determined by the  current date. In first
two days of every  month, two URLs are  created from every  definition
with  special  strings:  one  for  the  current  month and one for the
previous. In all other days just the current month is used, i.e.:

definition:
gopher://uninformativ.de/1/twitpher/#yyyy#-#mm#/

is parsed on 2017/08/01 and 2017/08/02 as:
gopher://uninformativ.de/1/twitpher/2017-07/
gopher://uninformativ.de/1/twitpher/2017-08/

but on 2017/08/03, 2017/08/04, etc. as:

gopher://uninformativ.de/1/twitpher/2017-08/

This is  just to  make  sure,  that the last post of the  month is not
missed,  even  though in my  timezone is  another  month  than in  the
timezone of the phlog author.

Then   the  script  goes  through  all the  parsed  URLs,  fetches the
gophermap and calculates checksum for each of them. If the checksum is
different than the last one stored, it makes an entry to the Bongusta!
gophermap and saves the new checksum.

And that's it. It could for sure be done  better, but it works for the
moment. I  still think that there is not much to see in the code, I'll
try to make it a bit  readable and then  share it with the rest of the
world. Right now I am  happy, that  Bongusta! works and that  somebody
finds it useful.

[1] gopher://i-logout.cz/1/en/bongusta
[2] http://github.com/logout128
[3] gopher://uninformativ.com/1/twipther

You are viewing proxied material from i-logout.cz. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.