Reddit is upping its AI game again, this time with the implementation

Reddit is upping its AI game again, this time with the implementation
of an LLM-powered harassment filter for the benefit of its army of
volunteer moderators.

The existence of the new Harassment Filter was discovered during an
[1]APK teardown by Android Authority, which spotted its existence in
version 2024.10.0 of the official Reddit app for Android. That version
of the official Android Reddit app has appeared in various unofficial
sources online, but the Google Play Store still shows 2024.08.0 as the
most recent generally available version.

The app contains lines of code referencing a large language model
trained to assist moderators in preventing publication of harassing
posts, and is backed up by a Reddit [2]help page updated last week that
reflects the existence of the new tool.

"The Harassment Filter is an optional community safety setting that
lets moderators automatically filter posts and comments that are likely
to be considered harassing," the help page reads. "The filter is
powered by a Large Language Model (LLM) that's trained on moderator
actions and content removed by Reddit's internal tools and enforcement
teams."

The filter can be enabled in a Reddit community's mod tools, but
individual moderators will need to have permissions to change subreddit
settings to enable it.

The harassment filter can be set to low ("filters the least content but
with the most accurate results") and high ("filters the most content
but may be less accurate"), and also includes an explicit allow list to
force the AI to ignore certain keywords, up to 15 of which can be
added.

Once enabled, the filter creates a new tag in the moderation queue
called "potential harassment," which moderators can review for
accuracy.

Reddit's help page says the feature is now available on desktop and the
official Reddit apps, though it's not clear when the feature was added.
We asked Reddit for more info, but haven't heard back.

More like (A)IPO

Reddit, which is busily planning its IPO, has been doing everything it
can to drum up investor support ahead of its [3]imminent stock market
debut. As with all things to do with raising cash in 2024, that means
sticking AI anywhere it can.

The popular social media platform [4]signed a deal with a previously
unknown AI business last month to let its models slurp up user
conversations to train products, earning it $60 million for its
efforts. While that figure hasn't been explicitly linked to Google,
both Reddit and the Chocolate Factory announced an expanded partnership
days after news of the AI deal broke.
* [5]Thar be safe harbor: Reddit defeats third attempt to unmask
digital pirates
* [6]Reddit signs AI training deal with Google – and why OpenAI's
Altman could be the winner
* [7]Reddit: If you want to slurp our API to train that LLM, you
better pay for it, pal
* [8]Reddit confirms BlackCat gang pinched some data

[9]According to Reddit, the deal gave Google access to its Data API,
and through it "an efficient and structured way to access the vast
corpus of existing content on Reddit … including supporting new ways to
display Reddit content and providing more efficient ways to train
models."

Google, meanwhile, said that Reddit will gain the ability to integrate
Google's [10]Vertex AI "to enhance search and other capabilities on the
Reddit platform" as part of the deal. For those unfamiliar, Vertex is
Google's tool for integrating Gemini capabilities into enterprise
environments.

How Reddit users – a notoriously [11]unruly bunch – will react to news
that AI is being foisted on moderators isn't clear. Redditors haven't
necessarily [12]reacted well to the company's planned IPO, with some
referring to the plan as a "pump-and-dump" scheme to enrich Reddit
leadership at the expense of the site.

Reddit has experienced the effect of a blowback before, with
[13]thousands of the site's subreddits going dark last year in protest
of a plan to begin charging developers to use the site's APIs,
effectively killing many third-party Reddit clients.

Reddit knows its users are a risk – it even disclosed its concerns in
an SEC [14]filing last month regarding its IPO plans. Ostensibly a tool
to assist moderators, the addition of AI could also be seen as a subtle
threat – protest again and we'll simply replace you volunteers with
Gemini, at least until it gets trained on enough Reddit data to start
siding with the protesters. ®

References

1. https://www.androidauthority.com/reddit-ai-model-fight-harassment-apk-teardown-3422993/
2. https://support.reddithelp.com/hc/en-us/articles/23856209638932-Harassment-Filter
3. https://www.bloomberg.com/news/articles/2024-03-04/reddit-s-meme-stock-forum-users-threaten-to-bet-against-its-ipo
4. https://www.theregister.com/2024/02/20/reddit_content_ai_deal/
5. https://www.theregister.com/2024/02/13/reddit_pirates_win/
6. https://www.theregister.com/2024/02/22/reddit_google_license_ipo_altman/
7. https://www.theregister.com/2023/04/18/reddit_charging_ai_api/
8. https://www.theregister.com/2023/06/20/reddit_confirms_blackcat_extortion_attempt/
9. https://www.redditinc.com/blog/reddit-and-google-expand-partnership
10. https://cloud.google.com/vertex-ai
11. https://www.bloomberg.com/news/articles/2024-03-06/reddit-s-ipo-success-hinges-on-infamously-unruly-user-base
12. https://www.reddit.com/r/technology/comments/1ayve0f/a_lot_of_redditors_hate_the_reddit_ipo_reddit/
13. https://www.theregister.com/2023/06/12/reddit_api_outage/
14. https://www.sec.gov/Archives/edgar/data/1713445/000162828024006294/reddits-1q423.htm#i1b9a579e78a34dfa99f7f26daeec195b_2017