/~bencollver/log/2024-12-17-no-dns-no-problem on tilde.pink

	View source

	# 2024-12-17 - No DNS, No Problem

	The early Internet used /etc/hosts files to resolve names. These
	were distributed on FTP sites. You can see a 1992 example below.

	host.txt from textfiles.com

	The /etc/hosts file quickly grew too large to be manageable.
	See historical document below.

	hosts.hst from textfiles.com

	Internet users switched to DNS.

	Recently, for various reasons, i became interested in avoiding DNS,
	so i researched using an /etc/hosts file. I found one on
	gopher.quest, but unfortunately it is incomplete.

	hosts.txt at gopher.quest

	I noticed that the Quarry gopher search engine contains a little over
	300 live gopher hosts. I wrote the author about it, and he generously
	created an /etc/hosts file from the Quarry database, linked below.

	hosts.txt at gopher.icu

	This is updated daily. On a linux system, you could replace
	/etc/hosts with this file and assuming a sane C library, it should
	"just work." It will resolve those names without DNS.

	FreeDOS is slightly more complicated. Suppose the file is in
	C:/etc/hosts.txt and will be used by mTCP and WATTCP.

	For mTCP, edit mtcp.cfg, add:

	HOSTSFILE C:/etc/hosts.txt

	For WATTCP, edit wattcp.cfg, add:

	hosts = C:/etc/hosts.txt

	Now FreeDOS programs should resolve those names without DNS.

	This covers gopher hosts. I would also like to include FTP and web
	hosts referenced by my gopher maps. Below is a script i wrote to
	mirror gopher content.

	gopher-get.tcl

	tclcurl.tgz SlackBuild

	Using this script, i mirrored my gopher content.

	$ mkdir gopherhole
	$ cd gopherhole
	$ tclsh gopher-get.tcl --binary-skip --images-skip \
	--ignore gopher://tilde.pink/1/~bencollver/dir \
	--ignore gopher://tilde.pink/1/~bencollver/dict \
	--ignore gopher://tilde.pink/1/~bencollver/gamefaqs \
	--ignore gopher://tilde.pink/1/~bencollver/ia \
	--ignore gopher://tilde.pink/1/~bencollver/recipes \
	gopher://tilde.pink/1/~bencollver

	This took a while, so i let it run and came back later. This created
	two directories.

	* tilde.pink/0 (Plain text documents)
	* tilde.pink/1 (Gopher maps)

	I scraped the host names from these directories as follows.

	$ find tilde.pink/1 -type f -exec \
	grep -e URL:ftp: -e URL:http: -e URL:https: {} \; \|\
	sed -e 's,.URL:[^:]://,,' -e 's,/.*,,' >names-gopher.txt
	$ find tilde.pink/0 -type f -exec \
	awk -f setext-scrape-links.awk {} \; >names-setext.txt
	$ cat names-gopher.txt names-setext.txt \| sort \|\
	uniq >names-extra.txt
	$ wc -l names-extra.txt
	392 names-extra.txt

	setext-scrape-links.awk

	This shows that i reference approximately 400 non-gopher names. Now i
	would like to resolve these to IP addresses. I wrote a short AWK
	script to do this. I ignored unkown names because of old documents
	that reference defunct names.

	$ awk -f resolve.awk names-extra.txt >hosts-extra.txt
	$ wc -l hosts-extra.txt
	612 hosts-extra.txt

	resolve.awk

	Because DNS can resolve a name to multiple IP addresses, this list
	grew from 392 to 612 lines. I appended this list to the Quarry hosts
	file like so.

	$ curl -o hosts-icu.txt gopher://gopher.icu/0/files/hosts.txt
	$ cat hosts-icu.txt hosts-extra.txt >hosts-all.txt
	$ wc -l hosts-all.txt
	1058

	For FreeDOS, filter out ":" to exclude IPv6 addresses. Also, mTCP
	has an 80 character line length limit including EOL characters.

	$ grep -v : hosts-all.txt \|\
	awk 'length($0) < 78 {print}' >hosts-dos.txt

	Now i can copy hosts-all.txt to /etc/hosts
	or copy hosts-dos.txt to C:/etc/hosts.txt
	and i am good to go.

	p.s.

	The FreeDOS bundled mTCP has a bug. If you specify HOSTSFILE but no
	NAMESERVER, then name resolution will always fail. You can work
	around this by making sure mtcp.cfg has a NAMESERVER line. If you
	don't use DNS at all, it can be:

	NAMESERVER 127.0.0.1

	tags: bencollver,retrocomputing,technical,unix

	# Tags

	bencollver
	retrocomputing
	technical
	unix