i---------------------------------------- | |
iIdeas for an aggregator | |
iFebruary 06th, 2018 | |
i---------------------------------------- | |
i | |
iI want to build a new gopher aggregator, specifically for phlogs, | |
iand I want it to work in a very particular way. I'm not sure this | |
iis going to be a general solve for everyone, but if I can do it | |
iright it will work for my personal use. | |
i | |
iVersion 1 Plan: | |
i | |
i - It should know that a phlog has been updated, not a whole | |
i gopherhole. | |
i | |
i - I should provide it an array of gophermaps to keep track of. | |
i | |
i - The gophermap will be hashed each day--store the hashes in | |
i a file for simplicity, or a sqlite db-- and if the hash | |
i changed from the previous day, that link is added to the | |
i daily list. | |
i | |
i - If the link doesn't resolve, ignore. It might be temporary. | |
i | |
i - It should organize phlog updates by date so I can go back if | |
i I missed a couple entries (like Bongusta) | |
i | |
i - It should auto-update daily | |
i | |
i - Output should be simple: | |
i | |
i - Master gophermap is a symlink to the current month. | |
i | |
i - Each gophermap is a year-month for easy archival | |
i | |
i - Viewing the directory will list the archive and current | |
i | |
i - Linking to current will include a link to the archive | |
i (directory view) | |
i | |
iVersion 2 Plan: | |
i | |
i - The link added to the aggregated list should point to the new | |
i entry, not the main gophermap. | |
i | |
i - Potentially do a diff on the gophermap if the hash has | |
i changed and try to identify new links? | |
i | |
i - Grab any new links and drop them in verbatim, potentially | |
i | |
i - Relative links might need to be rewritten | |
i | |
i - Problems with changed links? | |
i | |
i - Would this be instead of linking to the map, or in addition | |
i (sub links under a heading?) | |
i | |
i - Allow a way for people to submit their phlog to the list and | |
i make it a public system instead of a private aggregator | |
i | |
iMy plan was to implement this in Perl as a way to learn Perl. | |
iI may still go ahead with that, but it might be easier for me to | |
ibuild with my current knowledge in python. Regardless, if anyone | |
ireading this wants to take a stab at it, go right ahead. I'd | |
irather have the tool than be the one to write it. | |
i |