i finally have something resembling data from when @SuricrasiaOnline posted a link to haha.business yesterday! (hi it's me, owner and proprietor of haha dot business)
the graph in the first image shows the next 15 minutes after the post was made, split out by mastodon instances in blue, non-mastodon fediverse instances in orange, and everything else in green.
the data in the second image shows the per-second request counts in the first 10 seconds after posting the link.
some thoughts in a thread 🧵
so what's going on here? when you link to a website like https://haha.business, most fediverse instance software will go out and request the site to see if it has opengraph tags or other data so it can show a site preview card. (in the case of haha.business it'll even make a second request to fetch the funny little guy on the phone)
a lot of sites do this, but the difference with the fediverse is that it's happening *hundreds* of times because each individual instance is going and fetching the page on its own to generate the data. this metadata isn't federated by the source instance (and even if it were, would other instances trust it?)
in fact, i recorded 1846 requests in the 15 minute period after that post from just fediverse instances. 709 of those requests came in the first minute, which is an average ~12 requests per second.
bearing in mind that each instance is sending two requests for the HTML and the preview image, that's still a sustained 6 or so requests per second for a page that might be dynamically generated from a database or some such. and if you've been on fedi for a while, you may have even noticed that some sites appear to go down shortly after linking them on the fediverse.
anyway some analysis. i split out mastodon and non-mastodon fediverse instances because mastodon instances will wait a random interval between 0 and 60 seconds before making a request. if it didn't, all of those 709 requests in the first minute would try to happen in the first few seconds. it appears non-mastodon software in the fediverse does not attempt to spread load.
the later spikes in the graph are probably from boosts of that post making it to instances that didn't see it immediately after posting.
we'll see if i write this up in a blog post this weekend or not. hopefully i will because i want to show off what i learned (not much) about pandas
i'm not sure what lesson to take from this. i am already kind of down on federated protocols because they're hard to design and even harder to not turn into an accidental DDoS machine, so it's probably unfair for to me impart some lesson from this.
it'd be neat if the preview card was federated data? but the threat modeling of "oh no an instance might fake the preview card data and users could get phished or something" vs "oh no ~500 fedi instances will all hit a site within a minute and it might go down" is weird
@iliana it’s a trust thing, right? not being able to trust the instances you are federating with is the same sort of thing (and this is a criticism) which drives bluesky folks to the blockchain
if instances could acknowledge trust (and many mastodon instances do informally trust each other) then they could not only supply each other with things like OG tags but actually fill in missing information which might not be provided at the source
i FEEL LIKE “let my instance say it trusts icosahedron.website not to be evil” was a solution proposed the last time this conversation went around but obviously that hasn’t gone anywhere in the time since then
