title: Responding to Screwtape's ChatGPT Essay
date: 2023-02-01
tags: phlog political-thought programming reading sdf
identifier: 20230201T083244
---------------------------
I was listening to Screwtape's Lispy Gopher Show again last
night. Part of the show was given over to a discussion about
ChatGPT and a piece he wrote on his phlog[1] highlighting several
problems with the current iteration.
There are two quite distinct parts to the piece. They seem some-
what disconnected and contradictory, but I think that's a misun-
derstanding due to missing context (which is the only actual
criticism I have really).
The first part discusses problems with ChatGPT, and GPT-3 in gen-
eral. It starts by imploring the reader to avoid interacting with
it or anything connected to it. This appears to be immediately
contradicted by helpful instructions for downloading the source
code[2] and research[3] for GPT-2, its predecessor.
The second part paints a (somewhat laboured) analogy to try to
make visceral the problems with the licensing model and motiva-
tions of its creators, OpenAI. It eventually emerges into a dis-
cussion of the logical endpoints of surveillence capitalism when
married with this notion of counterfeiting as artificial intelli-
gence.
I think that for whatever reason, some folks have taken these to
be contradictory and separate positions, but I think that's due
to Screwtape being inclined to think faster than he writes.
The thread running through is this: he's extremely interested in
the technology, and its applicability in a non-surveillance non-
capitalist context. This context was filled in during the show
last night. Knowing this, there is no contradiction between en-
treating users to avoid ChatGPT and pointing them to the original
work (and warning them against using Reddit a training set, for
reasons which should be blindingly obvious to any non-MAGA).
This piece is largely a piece of anti-surveillance capitalism
polemic, using the ChatGPT project as its central example and
cautionary tale. It has nothing at all to say about the quality
of the work, or its ethical implications as distinct from its
surveillance function. In fact, there is a basic interest and cu-
riosity in the work itself, which is hidden from view.
I agree wholeheartedly with the position Screwtape is taking in
regards to surveillance capitalism, capitalism and privacy. I
think the more interesting questions are ethical and concerned
with the particular and insidious ways in which machine learning
can be used to perpetuate power dynamics and provide a layer of
indirection, obfuscation, and "plausible deniability" to the own-
ers of the IP and the operation of derivative products. Screw-
tape's essay doesn't really touch on that subject.
Ultimately, I'm looking forward to reading about his adventures
in making use of this work in a private, decentralized way. There
is work going on all over the place to try to re-decentralize the
internet, and life in general. The fediverse project is a part
of that, which has been helpfully re-fuelled by Musk's decloaking
of Twitter. I'd love to see work like GPT being used in conjunc-
tion with projects like CHERI[4] and the CRDC[5] for example.
Finally, I'd just like to say, this place is awesome and you all
are too.
Footnotes
---------
[1]
gopher://beastie.sdf.org:7991/0phlogs/do-not-touch-chatgpt.txt
[2]
https://github.com/openai/gpt-2
[3]
https://openai.com/blog/gpt-2-1-5b-release/
[4]
https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
[5]
https://www.cl.cam.ac.uk/research/srg/projects/crdc/