this post was submitted on 18 Jan 2024
102 points (96.4% liked)

Fediverse

28519 readers
591 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!

Rules

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy

founded 2 years ago
MODERATORS
 

A few months ago I released the Defederation Investigator, a tool to verify the federation status of Lemmy instances. With this new update, I've expanded it to support multiple Fediverse softwares, including:

  • Mastodon
  • Misskey
  • Mbin
  • Pleroma & Akkoma
  • Friendica

This works both ways: you can verify which Mastodon (et al) instances have defederated your Lemmy instance, as well as check the federation status of an instance running any of the supported softwares.

Like most of my works, this tool is FOSS and available on my team's GitHub.

Limitations

Many microblogging platforms, Mastodon included, offer admins the possibility of hiding their blocklists from the public. As it turns out many instances have chosen this approach, so the available information can be pretty limited at times.

Also, this update has increased the pool of instances from a couple hundred to over 2 thousand, so query times have increased significantly. You can reduce them by deselecting some softwares from the query page (hint: most fedi instances are Mastodon ones, so by deselcting them you'll cut out over half of the pool).

What about Kbin?

To my knowledge, Kbin doesn't share its federation status through an API like most softwares do. Furthermore, given recent events, I have little faith in the Kbin project. Instead, I chose to support its community driven fork: Mbin.

What about Peertube and Pixelfed?

I tried looking through their API docs and wasn't able to find any endpoints sharing either federation or defederation statuses. If anyone is familiar with any of these softwares and has any ideas on what to do to retrieve such information feel free to contact me, I'd love to add support for both.

What about ...?

Want more softwares? Feel free to propose them. I'd like for this tool to support as many projects as possible.

top 34 comments
sorted by: hot top controversial new old
[–] mulcahey@lemmy.world 14 points 10 months ago (1 children)

Thank you for this great tool!

Is there a way I could use this to find a Lemmy instance that let's me interact with desired instances? For example:

  • I want to participate in Instance A, B, and C.
  • A has defederated from B and C
  • can I generate a list of instances that let's me interact with A, B, and C? Then I'll move my account there.

Thank you!

[–] Nerd02@lemmy.basedcount.com 6 points 10 months ago* (last edited 10 months ago)

Yeah sure. Assuming you are only targeting Lemmy instances (other softwares make this a bit more complicated), A "can interact" with B if:

  • A hasn't blocked B
  • B hasn't blocked A
  • Neither A nor B are on allowlist. If either is on allowlist, it must have explicitly added the other one to its allow list (this is very uncommon, the only big instance using allow lists is hexbear.net)

So, to verify this, you could query the Defed Investigator with the instances you care about, one at a time. Only select the softwares you care about (likely only Lemmy) to make the query faster. Say you wanted to verify the compatibility between lemmy.world and sh.itjust.works (just making an example). Go to https://defed.xyz/check?name=sh.itjust.works&software=lemmy

  • lemmy.world doesn't appear in the "Instances defederated from sh.itjust.works" (this means .world hasn't blocked SJW)
  • lemmy.world doesn't appear in the "Instances defederated by sh.itjust.works" (this means SJW hasn't blocked .world)
  • lemmy.world doesn't appear among the "Instances not allowing sh.itjust.works" (this means .world isn't on allowlist or, if it is, it has explicitly allowed SJW. Again, this is very uncommon)

Also make sure the instance you are looking for isn't among the "Instances that returned errors", of course.

[–] Oha@lemmy.ohaa.xyz 11 points 10 months ago* (last edited 10 months ago) (2 children)

Wouldnt it be smarter to query the blocked list on the backend like once a day instead of doing thousands of requests in users browsers?

[–] iso@lemy.lol 8 points 10 months ago

Well, that's probably requires a back-end server and a database.

[–] Nerd02@lemmy.basedcount.com 8 points 10 months ago

Like @iso@lemy.lol said, can't query from the backend if there is no backend :)

It's all serverless, so this is as good as it gets.

[–] Annoyed_Crabby@monyet.cc 5 points 10 months ago
[–] Blaze@discuss.tchncs.de 3 points 10 months ago
[–] kurumin@linux.community 3 points 10 months ago (1 children)

Hi is the automod and user flair projects still active?

[–] Nerd02@lemmy.basedcount.com 3 points 10 months ago (1 children)

Hi. Yeah, both are still active, up and running.

User flairs are currently invisible on my instance because I didn't get the chance to update my modded UI to the most recent Lemmy version, but the microservice itself is still perfectly operational.

[–] kurumin@linux.community 2 points 10 months ago (1 children)

Pretty cool man! Have you considered making a PR to include the flairs to lemmy repo?

[–] Nerd02@lemmy.basedcount.com 2 points 10 months ago (1 children)

I have, even Nutomic asked me to, but the thing is I'm a bit of a noob when it comes to Rust. That was my first Rust project ever and the Lemmy backend is big and scary. I would rather gain some proficiency with the language first. Plus, it would likely have to be structured somewhat differently than my implementation if it was integrated into Lemmy proper.

[–] kurumin@linux.community 2 points 10 months ago

Oh I get it, I am in the same stage of Rust myself. I want to contribute in the future though.

Neat thanks!

[–] originalucifer@moist.catsweat.com 2 points 10 months ago (1 children)

i de-federated the defunct artemis.camp

does not show up on the list for my mbin instance https://moist.catsweat.com

[–] Nerd02@lemmy.basedcount.com 2 points 10 months ago* (last edited 10 months ago) (1 children)

There seems to be some problem with your instance's API. While the federation page works fine, when polling the defederation API at

https://moist.catsweat.com/api/defederated

I am getting the following error:

{
    "type": "https://tools.ietf.org/html/rfc2616#section-10",
    "title": "An error occurred",
    "status": 500,
    "detail": "Internal Server Error"
}

this also gets interpreted as an error by my tool, which is why the green "Instances defederated by ..." sections is empty.

[–] originalucifer@moist.catsweat.com -1 points 10 months ago (1 children)

i am running the standard mbin, no mods. i suspect youd have the same issue with all mbin servers

[–] Nerd02@lemmy.basedcount.com 6 points 10 months ago (2 children)

Actually I don't. After having tested it on the biggest mbin instances, it worked fine on:

  • fedia.io
  • kbin.run
  • kbin.melroy.org
  • thebrainbin.org
  • rimworld.gallery
  • gehirneimer.de
  • mbin.0ut0f.space

However I did get the error on:

  • kbin.chat
  • kbin.life

So whatever it is, it's not just you. I'm afraid I can't investigate any further without looking at some instance logs, though.

[–] originalucifer@moist.catsweat.com 3 points 10 months ago (1 children)

Fixed! my instance api was broken due to a pathing issue which is resolved, and your tool now functions as expected. thank you!

[–] Nerd02@lemmy.basedcount.com 2 points 10 months ago

Nice! Glad you were able to find and fix an issue with your instance.

[–] originalucifer@moist.catsweat.com 2 points 10 months ago

interesting! now i need to know

[–] tekakutli@lemmy.dbzer0.com 2 points 10 months ago
[–] UndercoverUlrikHD@programming.dev 0 points 10 months ago (1 children)

0 defederated, 1 federated (lemmy.world) and 2091 errors?

[–] Nerd02@lemmy.basedcount.com 2 points 10 months ago (1 children)

Not quite, lol.

Some errors will happen, that's inevitable, CORS is a bitch, but if you are getting that many I'm going to make an educated guess and assume there's something wrong with either your device or your connection.

[–] UndercoverUlrikHD@programming.dev 1 points 10 months ago (1 children)

Tested my local instance, lemmy.world was the only one that returned as federated to programming.dev

[–] Nerd02@lemmy.basedcount.com 2 points 10 months ago* (last edited 10 months ago) (1 children)

Ah so you were querying programming.dev? In that case I got:
0 defederated from; 3 not allowing; 111 defederated by and 462 federated

Where are you running it from? Because... it works on my device.

[–] UndercoverUlrikHD@programming.dev 1 points 10 months ago (1 children)

Apologies, it was a mistake on my end. Tried it again and noticed uBlock going wild. It was blocking every website except for lemmy.world which was globally whitelisted in my settings.

[–] Nerd02@lemmy.basedcount.com 1 points 10 months ago (1 children)

Uh interesting. Did you have any fancy custom configuration? Maybe you were blocking all requests to other hosts or something like that? I also have ublock origin and it didn't give me any troubles.

[–] UndercoverUlrikHD@programming.dev 2 points 10 months ago

Nothing fancy, I simply block all third party content unless it's whitelisted.

[–] gabe@literature.cafe -3 points 10 months ago (1 children)

Oh my God. This is really, really, REALLY bad. I know this looks like a useful tool but extending this to software that isn't lemmy is going to make the rest of the fediverse enraged. Tools like these are used as active harassment vectors. Theres a reason why the other fediverse instances that make these tools are affiliated with kiwifarms. Seriously please consider disabling non-lemmy software interaction immediately!

[–] Nerd02@lemmy.basedcount.com 4 points 10 months ago (1 children)

I'm sorry could you please elaborate on why the rest of the Fediverse would be enraged, or how this could be used for harassment? I don't think I follow. I'll admit, I only interact with the Fediverse through Lemmy so maybe there's some dynamics of the Masto-sphere I'm not picking up.

My understanding is that Mastodon admins can choose to hide their /domain_blocks endpoint to either outside users or even to all non admins. (source), and as a matter of fact almost a thousand of the 1700 Mastodon instances I'm querying already do so, so really I can only get the federation status of the few hundred that remain.

I think the admins that prefer not to show their defeds, in fear of harassment, are already hiding them, so it should be ok for me to query the remaining ones.

[–] gabe@literature.cafe -1 points 10 months ago (1 children)

There are aspects of the wider fediverse culture that is ignored on lemmy, and thankfully its lack of interoperability has helped it avoid trouble (for now). I'm really not sure how to explain to you how it could be used for harassment as it requires a bit of detailed background to how the wider fediverse functions and its history and I don't really know if it would convince you if I did explain it in enough detail. Just because data is publicly scrape-able doesn't mean it's acceptable to do so.

[–] Nerd02@lemmy.basedcount.com 4 points 10 months ago (1 children)

Well of course I can't guarantee that I would be convinced, even after hearing that but explanation aside

Just because data is publicly scrape-able doesn’t mean it’s acceptable to do so.

Isn't it? If, an instance admin, has the possibility of hiding some data to the public and refuses to do so, it's either:

  1. Because they are fine with the public accessing it
  2. Because they are ignorant and unaware of such a feature, which I honestly don't think is an acceptable excuse (after all users have entrusted this person with their data, ffs)

At the end of the day what I am doing is nothing more than what any user could do by checking the "Moderated servers" section of the about page of any Mastodon instance.

I'm sorry but I'm really am not seeing the logic behind your point.

[–] gabe@literature.cafe 2 points 10 months ago (1 children)

Do you know what kiwifarms is? Just curious.

[–] Nerd02@lemmy.basedcount.com 3 points 10 months ago* (last edited 10 months ago)

I didn't, up until yesterday night when you mentioned it. Had a quick Google search and read the wikipedia page, holy fuck there's some sick people out there. But I still fail to see how defed.xyz could help them doxx or otherwise harass people.

I don't want to be the author of software used for harassment, obviously, but I don't think you could use my tool for that, even if you wanted to.