TITLE: Making a Gopherhole
DATE: 2019-11-20
AUTHOR: John L. Godlee
====================================================================


I have heard people on youtube, mostly DistroTube and Hex DSL,
talking about Gopher. I also saw it being discussed on Hacker News
once.

 [DistroTube]: https://www.youtube.com/watch?v=lUBhOgK5zQI
 [Hex DSL]: https://www.youtube.com/watch?v=ORgk-AwD7SQ
 [Gopher]: https://en.wikipedia.org/wiki/Gopher_(protocol)
 [Hacker News]: https://news.ycombinator.com/item?id=13855634

The youtubers and digitial minimalist / linux try-hards have
recently begun lauding Gopher as a sane alternative to browsing the
internet with HTTP and the World Wide Web, which they say are
bloated and beyond redemption since advertisers and data harvesters
have taken over.

Gopher is an internet protocol that was created in 1991. In my mind
it sits alongside FTP more than it does HTTP, in that it is merely
a way of organising file delivery in a heirarchical fashion. It
doesn't have the flexibility of HTTP, providing only plain text
formatted by certain conventions and a smattering of links to
connect files and pages. Some think this inflexibility is a good
thing, Gopher sites are: simple, transparent, text based (good for
the visually impaired) and consistent. In Gopher it's hard to
implement most of the more destructive mechanisms present on the
World Wide Web like user tracking, data harvesting and advertising.
Gopher sites are also phenomenally light on resources, providing
just plain text. It's not that HTTP can't be fast as well, but
there is a tendency to use all the features that the flexibility of
HTTP can afford. That being said, I think a lot of people are
confusing the World Wide Web with the internet. Gopher is an
internet protocol, but it satisfies a different set of needs to
HTTP and they shouldn't be compared so readily.

I set up a Gopher site (aka a Gopherhole) on the Super Dimension
Fortress, which provides its registered and validated users with
gopherspace running on the gophernicus server software for free. It
also provides lots of other services run through a shell account
that you can SSH into, such as POP3/IMAP mail, and IRC. They are a
good community of hobbyists which provides a valuable if
computationally limiting service. There are lots of other places to
host a Gopherhole and it's also possible to self-host, and there
are a few Gopher server softwares out there, the most popular right
now being pygopherd, I think.

 [Super Dimension Fortress]: https://sdf.org/
 [gophernicus server software]: http://www.gophernicus.org/
 [pygopherd]: https://github.com/jgoerzen/pygopherd

First I created a shell account, logged in, and validated my
membership with a $3 USD donation to the SDF by PayPal. My
validation came through in about 12 hours, but I imagine this
varies a bit. I SSH'd into my shell account with:

   ssh [email protected]

After entering my password I could then create a gopher with
mkgopher, which creates a directory ~/gopher. Inside this directory
is where I store all my gopher content. mkgopher also allows
various site level options to be configured, such as the site title
and the site description. It's also important to set file and
directory permissions so that other users can read the material in
the gopherhole. This can be done automatically within mkgopher with
the chmod command, or manually:

   find ~/gopher/ -type f -print0 | xargs -0 chmod 644
   find ~/gopher/ -type d -print0 | xargs -0 chmod 755

Gophermaps

At the root of ~/gopher there should be a file called gophermap.
This file defines the homepage of your gopherhole, with plain text
and links to reach other content.

An example gophermap looks like this:

   iWelcome to gopherspace /

   0This is a text file in a link  file.txt
   9This is a pdf file in a link   file.pdf
   1This is a link to a directory  subdir

   iSome more text.    /

   IAn image   img.gif

   0A file on another server   /gopher/relevance.txt
gopher.floodgap.com 70
   hA HTTP link to another server  URL:http://sdf.lonestar.org/

The numbers, and some letters are called itemtypes. They denote
what type of information the line holds. There are a bunch of
itemtypes, but I think the commonly used itemtypes nowadays are:

 Itemtype                   Content
 ---------- -----------------------
 0                        Text file
 1                        Directory
 7                     Search query
 9                      Binary file
 g                        GIF image
 h                         HTML URL
 i                      inline text
 s                       Sound file
 I                  Image (not-GIF)
 =            Execute shell command

One extra itemtype that I haven't found documented ANYWHERE is =.
This can be used to start a shell command on the server. For
example, to show the current date and time:

   =echo "`date`"

This could come in really useful if hosting a server on your own
machine, where you could point the shell to a script to do
basically anything you want. This makes gopher more extensible than
I originally thought. Maybe if Gopher had remained popular for a
long time, we might have ended up with gopher developers abusing =
to gather data on users and subversively advertise to us.

The full syntax of the whole line is, for a text file for example:

   0Description of
file<TAB>/path/to/content.md<TAB>domain.org<TAB>port_number

Note that <TAB> must be an ACTUAL tab character, not expanded to
multiple spaces as some text editors 'helpfully' adjust it. The 0
defines the line as pointing to a text file. Description of file
will appear in the page as a selectable link. /path/to/content
defines the path to the text file to be opened by the link. File
paths can be defined relative to the current gophermap, so if
file.txt is located in the same directory as gophermap the path can
just be file.txt. If the file is located on the same server as
gophermap, domain.org and port_number can be omitted. If the file
is on a different server, the server domain (domain.org) should be
added and the port number, which is usually 70. So if the address
of the remote file linked is
gopher://gopher.floodgap.com/0/gopher/relevance.txt, the domain
name is gopher.floodgap.com.

As a side note, I've found that in gopher browsers, if you want to
visit the rendered version of a gophermap, you can enter the
address like this, with a 1 after the domain:

   gopher://gopher.floodgap.com/1/gopher

If you want to view the unrendered version of the gophermap, you
can replace the 1 with a 0.

For a piece of text, the gophermap syntax is similar, but not
exactly the same:

   iSome text that will appear on its own<TAB>/

A single slash should be included after a single <TAB> following
the text to be displayed. An i should be included before the text.

I've found that the 9 itemtype can cover many non-standard
filetypes, as it basically just prompts the web browser to download
the file.

Directories can be nested below the top level gopherspace
directory. Each of these directories can have their own gophermap,
but they don't have to. In the top level gophermap the directory
can be called as:

   1This is a link to a directory<TAB>subdir

If there is a gophermap in subdir/ it will be opened. This
subdir/gophermap can contain relative links to files the same as
the top level gophermap.

There isn't a lot more to creating a simple functional gophermap,
the rest is just text formatting to design a well formatted page. I
like to split the top level gophermap into sections with headers
wrapped in == symbols, and to have an ASCII art header at the very
top of the top level gophermap. It's also customary to limit the
width of a gopher page to 69 characters, but I haven't found any
technical reason why this would be the case:

   i      _       _            _         _____           _ _
       /
   i     | |     | |          | |       / ____|         | | |
       /
   i     | | ___ | |__  _ __  | |      | |  __  ___   __| | | ___
___     /
   i _   | |/ _ \| '_ \| '_ \ | |      | | |_ |/ _ \ / _` | |/ _
\/ _ \    /
   i| |__| | (_) | | | | | | || |___ _ | |__| | (_) | (_| | |  __/
__/    /
   i \____/ \___/|_| |_|_| |_||_____(_) \_____|\___/
\__,_|_|\___|\___|    /


   iJohn L. Godlee /

   0Contact details    contact.txt
   0CV cv.txt

   i==== Phlog posts
================================================= /

   0Post 1 post_1.txt
   0Post 2 post_2.txt

   i==== Recent recipes
============================================== /

   0Mac and cheese mac_cheese.txt
   0Pizza dough    pizza_dough.txt

Browsers

To browse Gopher pages the Lynx browser in the terminal is the most
common way I think. Otherwise there are plugins for some web
browsers, like the Overbite plugin for Firefox. There are also a
few different online Gopher to HTML proxy services where you type
in the gopher address and the output is rendered in HTML, e.g.
GopherProxy, Floodgap's proxy, or Gopher Commons.

 [Lynx]: https://lynx.invisible-island.net/
 [Overbite plugin for Firefox]:
https://gopher.floodgap.com/overbite/
 [GopherProxy]: https://gopherproxy.meulie.net/
 [Floodgap's proxy]: https://gopher.floodgap.com/gopher/gw
 [Gopher Commons]: https://gopher.commons.host/

I like to use the w3m browser which I couldn't get to load gopher
pages by default, but I did find an awk script on Bitbucket which
converts Gopher to HTML and then serves it through w3m. So far it
is working pretty well, but I still found myself loading up Lynx
when I was building my page initially to make sure everything was
formatted correctly.

 [w3m browser]: http://w3m.sourceforge.net/
 [awk script on Bitbucket]:
https://bitbucket.org/iamleot/gopher2html/src

Converting a Github-pages blog to Jekyll

I created a shell script which generates a gopherhole with a
directory tree like this:

   .
   ├── contact.txt
   ├── cv.txt
   ├── gophermap
   ├── posts
   │   ├── 2017-07-20-ranger-rifle-conf-mac.txt
   │   ├── 2017-08-14-bash-prompt.txt
   │   ├── 2019-11-10-beamer.txt
   │   ├── 2019-11-15-gginext.txt
   │   └── gophermap
   └── recipes
       ├── Apricot_orange_blossom_baklava.txt
       ├── Baked_pumpkin_with_apple.txt
       └── gophermap

It's basically just a load of shell scripts. First I use pandoc to
convert the recipes and blog posts to plain text with something
like:

   pandoc --from markdown --to plain --reference-links
--reference-location=block -o posts/post_1.txt post_1.md

Then I put the title of each blog post as a link into the gophermap
using sed in a for loop, with something like this:

   all=(posts/*.txt)

   # Reverse order of posts array
   for (( i=${#all[@]}-1; i>=0; i-- )); do
       rev_all[${#rev_all[@]}]=${all[i]}
   done

   # Get 10 most recent posts
   recent="${rev_all[@]:0:10}"

   # Add recent post links to gophermap
   for i in $recent; do
       line=$(head -n 1 $i)
       printf "0$line\t$i\n" >> gophermap
   done

Those were just simplified examples, it's easy to add header
material from another file or format the link text differently.

Update 2019_12_25

Note that my gopher hole is now hosted on tilde.club/, at
gopher://tilde.club/1/~johngodlee because scp was being problematic
on SDF. I've since shut down my SDF gopherhole.

 [tilde.club/]: http://tilde.club/

Update 2020-03-28

Note that my gopher hole is now hosted on
republic.circumlunar.space/, at
gopher://republic.circumlunar.space/1/~johngodlee because
tilde.club kept being unavailable. I'll be shutting down my
tilde.club gopherhole soon.

 [republic.circumlunar.space/]: https://republic.circumlunar.space/

You are viewing proxied material from republic.circumlunar.space. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.