* * * * *

    It's no longer possible to write a web browser from scratch, but it is
               possible to write a gopher browser from scratch

As I mentioned two months ago [1], I've been browsing gopherspace. At the
time, I was using an extension to Firefox [2] to browse gopherspace, but a
recent upgrade to Firefox left it non-working. I could use Lynx [3] but it
includes unnecessary pauses that make it feel slower than it really should
be. I also don't care for how it lays out the page.

So I've been writing my own CLI (Command Line Interface) gopher client.

And it's not like the protocol [4] is all that difficult to handle and
everything is plain text.

How hard could it be to download a bunch of text files and display them?

The protocol? Trivial.

Displaying the pages? Er … not so trivial.

The first major problem—dealing with UTF-8 [5]. Problem—the terminal window
can only display so many characters per line (default of 80 usually). There
are two ways of dealing with those—one is to wrap the text to the following
lines(s), and the other is to “pan-and-scan [6]”—let the text disappear off
the screen and pan left-and-right to show the longer lines. Each method
requires chopping text to fit though. With ASCII, this is trivial—if the
width of the temrinal is N columns wide, just chop the line at every N-bytes.
This works because each character in ASCII is one byte in size. But
characters in UTF-8 take a variable number of bytes, so chopping at arbitrary
byte boundaries is less than optimum.

The solution may look simple:

-----[ Lua ]-----
-- ************************************************************************
-- usage:       writeslice(left,right,s)
-- descr:       Write a portion of a UTF-8 encoded string
-- input:       left (integer) starting column (left edge) of display screen
--              right (integer) ending column (right edge) of display screen
--              s (string) string to display
-- ************************************************************************

local function writeslice(left,right,s)
 local l = utf8.offset(s,left)      or #s + 1
 local r = utf8.offset(s,right + 1) or #s + 1
 tty.write(s:sub(l,r - 1))
end
-----[ END OF LINE ]-----

but simple is never easy to achieve. It took a rather surprising amount of
time to come up with that solution.

The other major problem was dealing with the gopher index files. Yes, they
are easy to parse (once you wrap your head around some of the crap that
presents itself as a “gopher index” file) but displaying it was an even
harder problem.

Upon loading a gopher index, I wanted the first link to be highlighted, and
use the Up and Down keys to select the link and then the enter key to select
a page to download and view. Okay, but not all lines in a gopher index are
actual links. In fact, there are gopher index files that have no actual links
(that surprised me!). And how do I deal with lines longer than can be
displayed? Wrap the text? Let the text run off the screen?

At first, I wanted to wrap long lines, but then trying to manage highlighting
a link that spans several lines when it might not all be visible on the
screen (the following lines might be off the bottom, for instance) just
proved too troublesome to deal with. I finally just decided to let long lines
of text run off the end of the screen just to make it easier to highlight the
“current selection.” Also, most gopher index pages I've come across in the
wild generally contain short lines, so it's not that much of a real issue
(and I can “pan-and-scan” such a page anyway).

For non-text related files, I farm that out to other programs via the mailcap
[7] facility found on Unix systems. That was an interesting challenge I will
probably address at some point.

There are still a few issues I need to address, but what I do have works. And
even though it's written in Lua [8] it's fast. More important, I have
features that make sense for me and I don't have to slog through some other
codebase trying to add an esoteric feature.

And frankly, I find it fun.

[1] gopher://gopher.conman.org/0Phlog:2018/11/13.2
[2] https://www.mozilla.org/
[3] https://lynx.browser.org/
[4] https://www.ietf.org/rfc/rfc1436.txt
[5] https://en.wikipedia.org/wiki/UTF-8
[6] https://en.wikipedia.org/wiki/Pan_and_scan
[7] https://en.wikipedia.org/wiki/Media_type#Mailcap
[8] http://www.lua.org/

Email author at [email protected]

You are viewing proxied material from gopher.conman.org. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.