Well, I already posted today, so I'll post this one for
tomorrow. It's tomorrow already in most of the world
anyways.
I finished the automated website conversion script.
On my server, I put the script in /opt/<website_name>
Then I created a folder at /var/gopher/<website_name>. The
script turns all of the links on the website page into text
files and then creates a gophermap.
You can check out the results at:
gopher://gopher.visiblink.ca
Here's the script:
#!/bin/bash
# Clear the directory before starting (so that you can run the script as a cron job and update it at periodic intervals)
rm /var/gopher/LXer/*
# Get a list of links from the LXer page and save the list to a working file
lynx --dump
http://lxer.com/module/newswire/mobile.php | awk '/http/{print $2}' | grep http > /var/gopher/LXer/working_file.txt
# Dump the link pages to text files with usable filenames (that is, without the slashes):
for i in $( cat /var/gopher/LXer/working_file.txt ); do lynx --dump -nonumbers -nolist -hiddenlinks=ignore -width 60 $i > /var/gopher/LXer/"${i////_}"; done
# Dump the usable filenames to a text file
for i in $( cat /var/gopher/LXer/working_file.txt ); do echo "${i////_}" >> /var/gopher/LXer/usable_filenames.txt; done
# Generate a file with actual web page titles (preceded by a zero for the gophermap)
for i in $( cat /var/gopher/LXer/working_file.txt ); do wget -qO- ${i} | perl -l -0777 -ne 'print 0,$1, if /<title.*?>\s*(.*?)\s*<\/title/si'>> /var/gopher/LXer/titles.txt; done
# Create a gophermap by merging the files ("paste" merges the lines of each file together, with a tab between them, which is super convenient of it to do!)
paste /var/gopher/LXer/titles.txt /var/gopher/LXer/usable_filenames.txt > /var/gopher/LXer/gophermap
