Introduction
Introduction Statistics Contact Development Disclaimer Help
_______ __ _______
| | |.---.-..----.| |--..-----..----. | | |.-----..--.--.--..-----.
| || _ || __|| < | -__|| _| | || -__|| | | ||__ --|
|___|___||___._||____||__|__||_____||__| |__|____||_____||________||_____|
on Gopher (inofficial)
Visit Hacker News on the Web
COMMENT PAGE FOR:
Why AO3 Was Down
CubsFan1060 wrote 5 hours 11 min ago:
I didn't see it mentioned, but the quick fix for this (assuming you
don't depend on the order of id's) is just to alter your sequence to
use the max negative int, and increment from there. Not a complete
solution, but buys enough time to actually fix the issue.
olivermuty wrote 9 hours 38 min ago:
Uh, the bookmark that broke it all was to a part of the internet I have
yet to experience since getting online some 30 years ago. Alphas and
betas and omegas, it was a wild ride.
Freak_NL wrote 4 hours 54 min ago:
The bookmark itself, for the curious: [1] That alpha/beta/omega thing
is quite huge apparently, but not something you would ever encounter
outside of specific subcultures (like Archive of Our Own):
[1]: https://archiveofourown.org/bookmarks/2147483647
[2]: https://en.wikipedia.org/wiki/Omegaverse
p0w3n3d wrote 10 hours 48 min ago:
Hacker News helps me everyday break my information bubble. Archive Of
Our Own is something that I wouldn't walk into when wandering through
the internet
MangoToupe wrote 6 hours 50 min ago:
> Archive Of Our Own is something that I wouldn't walk into when
wandering through the inter
Kind of crazy to hear. AO3 is so culturally massive I don't know how
you miss it.
malnourish wrote 6 hours 15 min ago:
I have never heard of it, and I've been pretty active on the
Internet since the early 2000s.
bananaflag wrote 7 hours 3 min ago:
It is currently pretty much the main repository of fanfiction.
I've been reading fanfiction on the Internet for two decades, so for
me it would've been quite hard to miss it.
chii wrote 10 hours 36 min ago:
> I wouldn't walk into when wandering through the internet
it's interesting that some people are on the internet but is very
well insulated! AO3 is very well known for me...
paganel wrote 7 hours 12 min ago:
First time I read about it, not exactly sure what it is, by quickly
glancing at it. Looks like a collection of links to some
fan-fiction stuff.
Sharlin wrote 8 hours 22 min ago:
I’d say AO3 is the insulated part.
Having been an active internet user for longer than most AO3 users
have been alive, the first time I heard about it was a few years
ago in a student radio show about the fanfic genre and culture.
Poorly written smut featuring popular culture characters has just
never been my thing. Probably because I’m not that much of a fan
of any specific fictional setting or franchise in the first place.
xeonmc wrote 6 hours 9 min ago:
I think in this case, the more appropriate adjective would be
“quarantined”.
diggan wrote 9 hours 8 min ago:
> it's interesting that some people are on the internet but is very
well insulated
Not sure I'd call it "insulated", the internet is just very, very
vast, even when considering "just" the English-speaking web. Then
you have all the other "versions" out there too that are kind of
hidden to most people :)
Anecdotal, but also first time I heard about AO3, and I'd consider
myself having broad interests and generally well-read, although my
interests doesn't include fanfiction so maybe not so weird I
haven't heard about it before.
jorvi wrote 8 hours 38 min ago:
Its very much a gendered thing. If you have lots of female
(online) friends and late night topics with them ended up
trending spicy, you might hear of AO3.
FWIW the vast majority of writing on there is decidedly mediocre.
There is also an even more inferior alternative called Wattpad.
Funnily enough you learn that in general we aren't all that
different in our tastes, it's just that what men like to watch,
women like to read / imagine.
Edit: to paint the picture, this[0] was sent to me a while back
:-)
[0]
[1]: https://www.tiktok.com/@alexarowe11/video/74684621463476...
ethbr1 wrote 4 hours 39 min ago:
The great thing about crowdsourced content is that if you have
mediocre at scale, there are some gems!
I remember when I first stumbled across the main Antimemetics
Division storyline on SCP.
[1]: https://scp-wiki.wikidot.com/antimemetics-division-hub
diggan wrote 8 hours 7 min ago:
The world of "spicy reading" isn't new to me (male), just that
website in particular.
I don't think it's as gendered as you paint it, but I'd also
acknowledge it depends a lot on geographic location, probably
looks different where I am compared to where you are, I agree
with that we probably aren't all that different in tastes in
general :)
Freak_NL wrote 4 hours 39 min ago:
Anecdata, but it does seem like a very gendered divide on the
whole. The only reason I know of Archive of Our Own is
because my wife is quite familiar with it. And I do consider
myself well acquainted with various smut filled corners of
the internet well beyond Literotica.
parlortricks wrote 9 hours 49 min ago:
this is the first i've heard of it
zerocrates wrote 11 hours 5 min ago:
Is it faster to convert a column like this to unsigned? Obviously
assuming you don't use negative IDs in the application.
That's much more of a "kick the can down the road" solution to only
double your usable range, but if all positive the values in the rows
shouldn't actually have to change, just the column metadata, so it
could theoretically be more or less instantaneous. I guess in practice
this doesn't happen; the server would rather use its generic "rebuild
the table" alter method for changing a column type.
But it seems like you could reasonably do it if it's a
signed-to-unsigned change and there's no negative values and there's an
index on the column to make checking that fact fast. Or one of those
third-party/lower-level type tools could let you do it without any
checking.
afandian wrote 7 hours 33 min ago:
I don't know what DB was used in this csae, but Postgres doesn't have
unsigned integers. It always struck me as hugely wasteful, as e.g.
sequences start at zero by default.
masklinn wrote 5 hours 47 min ago:
At $dayjob we've actually used this property once or twice: if you
need to merge two tables you can keep the positive ids for the
first one, and use negative ids for the second one. It only works
once, but damn if it's not effective when you need it, and it
conveniently flags all the records with an id under some limit
(positive and negative both) as "pre-transition" record when you're
looking for patterns.
afandian wrote 2 hours 18 min ago:
I’ve also seen positive and negative ids for entities with
different properties (can’t recall what). Felt like an
unnecessary hack though.
adamcharnock wrote 9 hours 55 min ago:
An interesting idea! I suspect a major speed up would come from the
fact that the column is staying the same size. So (I assume) far
fewer bytes would need to be moved around.
12_throw_away wrote 11 hours 25 min ago:
For anyone who feels like looking up exactly what this bookmark was
pointing to: I did, and very much wish I hadn't!
nikanj wrote 9 hours 15 min ago:
I know I'll regret this, but how do you navigate to bookmark by
column id?
camel-cdr wrote 9 hours 9 min ago:
The link is in the reddit comments.
It's Dead Dove though.
Sharlin wrote 8 hours 20 min ago:
Translated from fanfic to English: "a warning tag that signifies
the story contains potentially disturbing or morally questionable
content."
nikanj wrote 8 hours 22 min ago:
Oh my. Well, today I've learned something new about gen z and the
internet
rsynnott wrote 7 hours 59 min ago:
AO3 is nearly 20 years old, and I think it was mostly
LiveJornal refugees anyway. This is one that you very much
can’t pin on gen-z.
eknkc wrote 9 hours 50 min ago:
What in the name of fuck
MangoToupe wrote 6 hours 48 min ago:
New to the internet?
heavensteeth wrote 7 hours 5 min ago:
I ask in complete earnest: is that your honest reaction to seeing
it, or did you hype it up for your comment? Personally very little
could evoke that kind of reaction from me. Maybe a little, "oh,
that's an interesting thing to be turned on by" but for the most
part, who cares?
Freak_NL wrote 4 hours 44 min ago:
I mean, it's fiction. If a writer can't explore the depths of
human behaviour there, then where?
The only uncomfortable thing there are the explicit references to
Harry Styles and Louis Tomlinson. I do take exception to using
real people in fiction if you proceed to heap abuse on the
characters which you model on those celebrities. (The story seems
to use only the given names, but the tagging makes the link
explicit.)
Obviously, you can refer to real world famous people in fiction
— it would be silly to write a book about 2025 America and not
mention that the president is Trump if it includes political
themes — but there are limits.
ThrowawayTestr wrote 6 hours 41 min ago:
I can't remember the time when I was so innocent that forced
mpreg breeding wasn't shocking.
Groxx wrote 11 hours 28 min ago:
Ha, a site I worked on hit this limit for the "follow relationships"
table - had to build a new compound key table to migrate to, with
triggers to dual read/write, to unbreak everything. In a few hours of
"wtf" -> "oh crap" -> "well I guess we gotta do it right this time" and
quick coding.
And then I pulled apart PT-OSC to make it more... less incredibly
stupid about resource use, so it wouldn't cause too much load while it
backfilled. And let it run for about 6 weeks.
Good luck! It's a fun problem to have - excess success, and a light
puzzle to solve :)
bilka wrote 5 hours 18 min ago:
Do you happen to have those PT-OSC changes around? We've already
migrated bookmarks with the downtime (with PT-OSC), but there are
more tables that would be nice to get migrated away from int without
going into maintenance or shedding a lot of load.
Groxx wrote 4 hours 5 min ago:
No, it's long, long gone.
When I did it, the script was a bit of a mess of trigger setup, and
then a backfill that only monitored replica lag, as if the status
of the much less heavily used failover instance was somehow the
most important part of a database. Hopefully that's no longer
true, and none of this is necessary any more.
So I essentially split it in half, so I could keep only the trigger
setup, and carefully read the queries the backfill would perform so
I could duplicate it. And then wrote a very simple loop of "select
N records, copy to new table, check how long that took. scale up by
min(5%, 100), scale down by 30%, if outside target bounds".
Intentionally very polite to the main DB, because once the triggers
are in place it really doesn't matter how long it takes. It dropped
down to single digits at peak load on some days, so I think that
was the correct choice.
madaxe_again wrote 11 hours 34 min ago:
This is like seeing a brick wall 40 miles down a straight road and yet
still managing to drive into it, and then blaming the wall.
alt187 wrote 6 hours 52 min ago:
> This is like seeing a brick wall 40 miles down a straight road and
yet still managing to drive into it, and then blaming the wall.
Not really, no. For example, if you drive into the wall, you may die.
Another experience that feels like death is working in a company that
implements on-call rotations.
It would be too easy to draw out a parallel between how you approach
a free fanfiction website (the website should mystically owe you five
9's uptime) and the mentality that metastased in the industry.
Instead, I'm gonna take this opportunity to point out that the AO3
downtime affected you, as a non-user, enough to vitrify the admin,
where hardcore users laughed it off (because they're not entitled
toddlers).
madaxe_again wrote 5 hours 34 min ago:
I don’t think I turned the admin into glass, nor vilified them -
just pointed out that this sort of thing is readily avoided.
But sure, I committed a hate crime.
randallsquared wrote 5 hours 36 min ago:
> enough to vitrify the admin
Not sure it was that solid.
ohdeargodno wrote 10 hours 45 min ago:
Ao3 doesn't have a dude getting slack alerts by a dozen monitoring
agents. It's one of the last holdouts of the old, more personal
internet. Hell, it's even certain that they forgot or even didn't
know that the type was an unsigned int.
And that's perfect. Blame the wall too, because it was running just
fine. It's a site to write (mostly porn), with better uptime and more
daily users than most of the companies posted on HN daily.
camel-cdr wrote 10 hours 17 min ago:
I wasn't sure what the percentage of porn is, so I counted the
number of works for each maturity rating:
4,247,583: Teen And Up Audiences
4,173,082: General Audiences
2,816,083: Explicit
2,271,446: Mature
1,676,061: Not Rated
darkwater wrote 11 hours 8 min ago:
I guess that whoever maintains that infra simply hadn't thought of it
or was not aware. It's not something you get for free in a monitoring
system with some agent like disk usage for example. You need to know
and remember you have a hard limit on IDs and be aware at which ID
you are.
hinkley wrote 8 hours 22 min ago:
Meanwhile if I keep reminding people where the wall is and how fast
we are approaching it I’m considered “negative”. That”…
real reason this stuff happens. If someone noticed, the got tired
of harping on it and without the constant barrage everyone else
immediately let it go out of sight, out of mind.
charcircuit wrote 11 hours 47 min ago:
>to fix it they have to migrate the entire database to use a different
type for bookmark IDs... except of course this will take a while
because there are two Billion Of Them Lol
You can shard them between 2 tables. Then migrate them to a single one
later.
ohdeargodno wrote 10 hours 42 min ago:
There's no SLA for Harry Styles porn. Run the migration, lock the
table for two days and redo the same in 13 years when you get to 4
billion bookmarks.
rsynnott wrote 7 hours 57 min ago:
I mean I’d assume they went for a 64bit integer. In a few million
years, people who are into weird porn about whatever the temporally
local equivalent of Harry Styles is (probably some sort of robot)
will once again be mildly inconvenienced.
camel-cdr wrote 10 hours 10 min ago:
> There's no SLA for Harry Styles porn
But what about my good night's sleep? How can I go to bed without
reading about my favorite blorbos?
ohdeargodno wrote 9 hours 44 min ago:
Real ones use bookmarks to find them ag- ah, shit.
Real ones back them up in a single .txt file
kijin wrote 10 hours 31 min ago:
In 13 years, the Unix timestamp will probably be a much bigger
problem.
RainyDayTmrw wrote 12 hours 40 min ago:
It's kinda impressive that they got to 2 billion rows - with indexes,
no less - without falling over.
jiggawatts wrote 9 hours 12 min ago:
Point queries — typical of this kind of app - scales as log(n) in
the number of rows. (Assuming a typical b-tree database index.)
This kind of workload cheerfully “scales” to your disk capacity.
notorandit wrote 12 hours 44 min ago:
> typical database column
Typical for 70s and 80s.
Honestly, designing a 21st century database is a different thing if
compared to back then.
You can use 128 bit integers, provided that you really want to use
integers.
And maybe you put a timestamp along.
rsynnott wrote 7 hours 54 min ago:
The website appears to date from 2008. This was a _very_ common
latent bug at that point, particularly because Rails would basically
force you to implement it. I assume this got fixed at some point, but
for a long time all ActiveRecord models had an autoincrementing ID,
which had to be a signed 32 bit int. There were scary monkey-patching
workarounds if you wanted something more sensible.
EDIT: And, yes, it is apparently Rails!
[1]: https://fanlore.org/wiki/Archive_of_Our_Own#Timeline
Sharlin wrote 8 hours 18 min ago:
One of the first things I internalized about databases was "just
always use BIGSERIAL for primary keys". There are very few good
reasons not to.
looperhacks wrote 4 hours 28 min ago:
Maybe don't:
[1]: https://wiki.postgresql.org/wiki/Don't_Do_This#Don.27t_use...
shakna wrote 9 hours 57 min ago:
Salesforce is a rather popular platform.
Its defaults are also either a 18-character ID, or a 32bit integer.
So, unless you take the effort to actually fight Apex, you're gonna
hit this problem sooner or later.
quickthrowman wrote 3 hours 30 min ago:
Doesn’t an 18-character alphanumeric ID give you 18^36
combinations? 1.54 x 10^45 seems like enough combinations.
shakna wrote 26 min ago:
That's the point of the "or". You probably don't know which
you're getting. It's what makes that particular design decision
bite you more often.
throwawaysoxjje wrote 10 hours 25 min ago:
Nah I made the same mistake back in 2009 for a system that was
storing behavior events during malware analysis.
You don’t often expect to have two billion of something until you
do.
9dev wrote 9 hours 57 min ago:
It's not like those two billion things just materialise in your
database, right? Someone must have watched that graph climb, and
climb, and climb, approaching the limit.
detaro wrote 9 hours 48 min ago:
If they have that graph and remember the limit they choose 15
years ago... It's not something you think about constantly
running a mostly stable code-wise site.
jarofgreen wrote 11 hours 26 min ago:
or use UUID/GUIDS, many databases (eg PostgreSQL) and frameworks (eg
Django) support them.
dwedge wrote 11 hours 15 min ago:
Using uuids can cause lots of problems with indexing,
fragmentation, row size and index size
j16sdiz wrote 12 hours 30 min ago:
let's use 128bit integer and handle them like floats in php!
and maybe put a 32bit timestamp along and pretend it can somehow
store more than a 32bit integer can.
schoen wrote 13 hours 41 min ago:
A bookmark for every view of "Gangnam Style"!
[1]: https://arstechnica.com/information-technology/2014/12/gangnam...
wging wrote 12 hours 18 min ago:
That article was from 2014, it has many more views now (about 5.6
billion).
<- back to front page
You are viewing proxied material from codevoid.de. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.