(C) Alec Muffett's DropSafe blog.
Author Name: Alec Muffett
This story was originally published on allecmuffett.com. [1]
License: CC-BY-SA 3.0.[2]


censorship and interception – dropsafe

2021-11

Regular readers will know that from 2013-16 I was a software engineer for Security Infrastructure at Facebook, working as part of the (then) Protect & Care team which also included the (then) Site Integrity team.

This was an “at the coalface” role of user protection, safety, integrity and security, and I benefitted greatly from being faced with implementing actual code to meet, enable and (importantly) balance ethical and commercial interests, rather than being (e.g.) a data scientist who might get excited about fixing a social problem, with a worldview shorn of the un-envisioned and unintended negative consequences of practical implementation.

One of the things I particularly learned was that sometimes the ostensible prevention of “harm” — enforcing restrictions on the basis of “signals”: user metrics, heuristics, and behaviour — sometimes caused more badness than goodness to humankind as a whole, and it was wisest to refrain from it.

In police forces they refer to it as “discretion”. Of course employee churn meant that sometimes such undocumented wisdom was forgotten and needed to be rediscovered by a new generation of engineers, but the general principle is invariant.

“Yungkendee” is a creator on TikTok, and is documenting her heavily-evidenced age-16 “legal kidnap” from her mother, transporting her to a “Wilderness Experience” camp in Utah; I would like to share two videos from her in particular, this one is an introduction:

Introduction to the issue… Video #1

But more important from a civil liberties perspective is the next video; not merely does it document some atrocious behaviour on the part of the “Wilderness” management, but also it highlights how young people use Instagram not only for personal expression but for mass communication with the people they care about — possibly in spite of their parents, but if these children are 16+ that too is a really interesting argument — and how this legitimate user behaviour could run counter to the calls to “protect” children from having a robust communication capability:

People using and sharing Instagram accounts… Video #2

So @Yungkendee, at their request, posted on the Instagram timelines of a couple of other girls who were in the same group as her, to let their friends know that they were okay; and when the “wilderness camp” program authorities learned this, they offered to re-kidnap her “for free” as means for her mother to pressure her into not doing it again.

Should Facebook have taken steps to stop Yungkendee?

There’s been recent argumentation, for instance from Frances Haugen — an ex-Facebook Data Scientist Product Manager, although the press repeatedly refers to her as a Data Scientist — that the ability of people to create more than one instagram account per person — is a threat to democracy:

…and there are US Senators who likewise perceive multiple-user access to accounts as somehow inhibiting the ability to protect users:

…but here’s the kicker: this is a SUMA. Just a small case, yes, but one which would be caught up in any prevention mechanism.

A very brute-force signal for this sort of thing would be enforceable as “once an Instagram application instance has been logged-in from a given phone, it can’t be reused for another account” — of course, in theory simply deleting and reinstalling the App should bypass this restriction, but there’s an entire subculture of application writers, loudly being cheered-on by the child-protection community, who are making an art-form of making GDPR-defying persistent data stores on Phones in a way that survives application reinstallation.[*]

So: watch the two videos above, and tell me in the comments below whether Facebook should have taken stronger steps (in line with calls from government, child safety experts, age-verification industry wonks, and the various Facebook whistleblowers) to prevent Yungkendee from being able to log into her friends’ Instagram accounts, and with their permission to post messages on their behalf.

I’ll be interested to hear your opinions. Comments below. More context at her TikTok account.

ps: My view? Hell, no.

[*] These data-stores would hold something like a “user age” or an “identifier” which can be use to spot somebody trying to “cheat” the “multiple-user-prevention” technology. Generally what happens is that such a value is poked into a hard-to-find and not-properly-garbage-collected entry in a shared “registry” on the device in question. I am saddened that more GDPR-fans are not hunting down and railing against these mechanisms.
[END]

[1] URL: https://alecmuffett.com/article/tag/censorship-interception
[2] URL: https://creativecommons.org/licenses/by-sa/3.0/

BoingBoing via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/alecmuffett/