Rethinking Moderation: Interview with Alexander Cobleigh

Rethinking Moderation: Interview with Alexander Cobleigh
=====================

As we all know, moderation is one of the internet's most difficult issues.
We've all seen the pattern: toxic behaviors manifest on our favourite
social media platforms & metastasize through them. We complain, moderators
struggle to keep up, & bad actors evade protective measures. Folks then
leave the platform or strike an uneasy peace as the toxicity becomes
endemic.

Alexander Cobleigh thought there had to be a better way. So he created a
new kind of moderation system: TrustNet
[https://cblgh.org/articles/trustnet.html]. Rusty chats with him about
this fascinating development.

Alex can be found on Mastodon: @[email protected]

---

QUESTION: Thanks for agreeing to do this email interview bit. You're doing
some fascinating work & I wanted a chance to pick your brain a bit more.

ANSWER: Of course! I'm glad that you think so & honoured that you asked.

Q: What got you interested in developing TrustNet as a project? What
motivated you to take on the work yourself?

A: It all started with Cabal, an open source peer-to-peer chat platform I
have developed with some friends, needing moderation capabilities. We
reached a point where we couldn't in good faith keep developing features
without the ability to remove potential malicious actors from a chat. At
the same time, I had also reached the point of my university studies that
my Master's thesis was coming up. So, really, the way it got started was
me thinking for a few months on how Cabal could have a moderation system
better than the individualistic naive solution of "well every single
person needs to individually block trolls", as well as wanting to work on
something worthwhile for my Master's thesis.

Q: The system seems like it may require a complex infrastructure. What
have been some of the challenges in trying to implement such a system?

A: It doesn't really require any complex infrastructure, really. What you
need is the following: some way for people to assign trust to others (i.e.
an interface), a way to store those trust statements, & a way to transmit
stored trust statements between participants

It would, for example, be possible to use TrustNet in a fork of Mastodon,
where a modified UI could let people of the Mastodon instance assign each
other as subjective moderators. The server hosting the instance would
receive and keep track of the trust issued by each person. A given
participant could then have a moderated experience through the moderators
they choose and trust, which could be different for different people
(pending on who they trust).

Of course, the current implementation is built with peer-to-peer systems
like Cabal or Secure Scuttlebutt in mind, but server-based paradigms can
just as well also make use of TrustNet.

The difficulties in developing TrustNet were in trying to represent
behaviour that made sense socially, while also making use of the system's
transitivity. The trickiest bit was coming up with a good solution on how
to partition the calculated subjective trust ranks where basically each
person in your trust graph is ordered according to their calculated trust
rank. The problem with the rankings is where to make a cut such that
everybody above the cut are regarded as trusted, and everyone below it as
not trusted (e.g. their moderation actions won't be automatically applied,
due to being too far away from you).

Q: In our era of tech dystopia, any kind of algorithmic ranking is
frightening to a lot of folks. What distinguishes a trust-based moderation
system from systems that assign a kind of "social credit score"?

A: The way I interpret algorithmic ranking with your mention of a social
credit score is from the following point of view of the problem: If
everybody assigns everyone else a trust score, then you have a popularity
contest where the people that manage to get the most trust, well, win (and
people without trust fall outside of society).

What this describes, however, is a *reputation* system. Reputation is an
aggregate, derived from the crowd. It can be used to inform trust, but it
is not trust. Reputation is "objective"; the reputation score for one
person looks the same no matter from which person's perspective you are
taking. Trust, on the other hand, is subjective. My trusted peers are
different from your trusted peers, which are different from a third
person's trusted peers.

Algorithmic ranking typically builds on machine learning, where you
increasingly dig yourself into you-shaped hole that is impossible to get
out of from the perspective of the ranking algorithm. The trust-based
approach I present in TrustNet is kind of a parallel route one can go down
to tackling similar problems, but where the end user is in control,
instead.

Q: I think one of the most fascinating aspects of this system is the
notion of the Trust Area, which as you state in your blog post, "captures
the context that the trust is extended within; for outside of the realm of
computers, we trust each other varying amounts depending on a given
domain." This makes total sense, but it's something I rarely see
considered in online platforms. What inspired that idea for you?

A: I wanted to avoid conflating trust within different areas, so that
TrustNet could be used for different purposes within the same chat system.
You might have one trust area, let's call it 'general purpose', that
controls whether people can send DMs to you, whether their profile images
should be visible, and whether the images they post should be
automatically shown or not. In the same system, you might want another
trust area to control who can hide or remove users and posts on your
behalf. If we consider these two trust areas, we can kind of get a feel
for the 'general purpose' trust area being less restrictive than the
'moderation' trust area.

After reading the computational trust literature more, my hunch on the
notion of a trust area was verified by it having been present in various
papers and research, albeit termed differently.

Q: I was wondering if you could talk a bit more about how you imagine an
average user interacting with this kind of system? How do you imagine them
interfacing with this system?

A: If we consider the moderation trust area, I basically think that
friends in a chat system like Secure Scuttlebutt would assign trust for
each other to delegate blocking responsibility.

It would look something like, going to one of your friends's profiles,
clicking a dropdown titled "Moderation Similarity" and picking one of the
options: None, Some overlap, Similar, Identical. Each option would
represent an increasing trust weight, which essentially controls the
impact of a trusted person's recommendations (that is, people whom *they*
trust for moderation). You don't need to do this for that many people for
it to start having an effect, maybe like 4-5 friends and that's all you'll
ever need.

On SSB, I have a feel for whom of my friends have a similar blocking
policy as my own (some may be too eager to block, for example).

Q: What will you be working on next with TrustNet?

A: Ah, that's a great question. The first thing that comes to mind is to
integrate it into Secure Scuttlebutt, where there is currently a lack of
any kind of delegated moderation system. The community has been very
receptive and encouraging of my thesis in general, and integrating it with
SSB, in particular.

I would also like to experiment with it further in Cabal, playing around
with a kind of mechanism for allowing greater privileges to people who are
trusted by my friends (or their friends). What I mean by that is, for
example, using TrustNet to limit which peers I will allow image posts or
avatars from. So, if someone is trusted at all from my perspective, my
cabal client would download and show their avatars, whereas for untrusted
peers a placeholder would be shown, instead. This limits the attack
surface of malicious actors like trolls or brigades.

Finally, it would also be fun to experiment more playfully with TrustNet
and see what comes out of that :)