2023-10-15 4chan client shell script

2023-10-15 4chan client shell script

I wrote this shell script a while back because I wanted to browse
the Lisp and Emacs general thread on 4chans /g/ board[1] in Emacs. I
thought about writing it in Elisp but handling Json data is a real
pain. So instead I use curl[2] to request a thread via the 4chan
API[3], then pipe it to jq[4] to process the result, then AWK[5] to
work its black magic to construct a very simplified html version of
the thread which I can finally view with Eww[6] in Emacs. Looking at
this code a few months later it kinda looks like the work of a
madman, though.

Features
~~~~~~~~

- produces a simple html file ideal for text browsers
- shows replies for every post (see screenshot)
- images are linked

Drawbacks
~~~~~~~~~

- if no thread number is provided it shows a list of threads from the
catalogue but ONLY those with a proper title
- the /g/ board is hard-coded (which can be easily changed of course)

Screenshot
~~~~~~~~~~

(lol) - Lisp General (M-x eshell Edition): file:///tmp/thread.html

[IMG] 96623294 Do 12. Okt 17:29:30 CEST 2023

How can I mentally think of C-x? It seems to do too much.
Replies: >>96623403 >>96623705

[IMG] 96623403 Do 12. Okt 17:34:58 CEST 2023

>>96623294
It's a sequence of keys, the control key and the x key
Replies: >>96623624

[IMG] 96623624 Do 12. Okt 17:45:21 CEST 2023

>>96623403
I know, but what do I call it?
Replies: >>96623807

-UUU: @%*- F2 *eww* 9% (165,0) (eww) -----------

Code
~~~~

#!/bin/sh

# if no thread number is provided
if [ $# -lt 1 ]; then
# print catalog, ONLY THREADS WITH TITLE (because I only need /lol/
# and printing the OP would be too much clutter, html and all)
# https://github.com/4chan/4chan-API/blob/master/pages/Catalog.md
curl -s https://a.4cdn.org/g/catalog.json | \
jq -r '.[].threads | map([.no, .sub] | join(" ")) | join("\n")' | \
awk '$2'
exit 1
fi

# pipeline: curl (thread.json) -> jq (thread.csv) -> awk (thread.html)
# https://github.com/4chan/4chan-API/blob/master/pages/Threads.md

curl -s https://a.4cdn.org/g/thread/$1.json > /tmp/thread.json

# extract title
jq -r '.[][0].sub' /tmp/thread.json > /tmp/thread.csv

# extract thread-number, unix time stamp, comment,
# unix timestamp + microtime (=filename), file extension
jq -r '.posts | map([.no, .time, .com, .tim, .ext] | join("\t")) | join("\n")' \
/tmp/thread.json >> /tmp/thread.csv

# black awk magic
awk '
BEGIN { FS="\t"; }
# first loop over file to get references
NR==FNR && match ($0, />>[0-9]*/) {
# e.g. match = >>91962953, (> is html code for >)
split(substr($0, RSTART, RLENGTH), to_arr, ";");

# e.g. x = 91962953 (= referenced post), no capture groups in mawk
x=to_arr[3];

# generate new html link
new_link = sprintf("<a href=\"#p%s\" class=\"quotelink\">>>%s</a>", $1 ,$1);

# replies[referenced_post] += referring_post (string concat)
replies[x] = sprintf("%s %s", replies[x] , new_link);
}
# second loop, write html
NR!=FNR && FNR==1 { printf "<html><head><title>%s</title></head><body>", $0; }
NR!=FNR && FNR >1 {
print "<table>";

# insert image link or placeholder
print "<tr><td>";
if ($4) printf "<a href=\"https://i.4cdn.org/g/%s%s\">[IMG]</a>", $4, $5;
else print "[IMG]";
print "</td>";

# insert post id, creation date of post in local time and comment
sprintf("date -d @%s", $2) | getline local_time
printf "<td>";
printf "<i id=\"p%s\">%s</i> %s <br><br> %s", $1, $1, local_time, $3;
print "</td></tr>";

# insert replies to this post
if (replies[$1]) {
print "<tr><td></td><td>Replies: ";
print replies[$1];
print "</td><tr>";
}

print "</table>";
}
END{ print "</body></html>";
}' /tmp/thread.csv /tmp/thread.csv > /tmp/thread.html

# open in *eww*
emacsclient --eval '(eww-open-file "/tmp/thread.html")'

Footnotes
~~~~~~~~~

[1] https://boards.4channel.org/g/
[2] https://curl.se/
[3] https://github.com/4chan/4chan-API/blob/master/pages/Threads.md
[4] https://github.com/jqlang/jq
[5] https://en.wikipedia.org/wiki/AWK
[6] https://www.gnu.org/software/emacs/manual/html_mono/eww.html