Likely within the API, but I would say be cautious as people on the fediverse really do not like having their data scraped, especially for projects like this.
this post was submitted on 15 Aug 2023
33 points (94.6% liked)
Fediverse
17534 readers
25 users here now
A community dedicated to fediverse news and discussion.
Fediverse is a portmanteau of "federation" and "universe".
Getting started on Fediverse;
- What is the fediverse?
- Fediverse Platforms
- How to run your own community
founded 4 years ago
MODERATORS
Interesting. Isn’t it entirely public though?
Yes, still doesn't make people fine with it. People defederate over stuff like that on mastodon. It's often seen as violating peoples privacy.
lol I won’t make you defend these people’s logic. Thanks for the warning!
Wow! Is that how little you care about other people?
What do you mean? I thought it was an explicit feature of this place that literally all of it was public and nobody owns any of the data. Isn’t it just sitting there in the public domain?
If people know that’s how it works, they can’t get mad if someone does access the data. Especially for innocuous curiosities like this.
The content of lemmy is in public, but is not in the public domain. They are different concepts.
Your public domain assumption doesn't have to apply to others, legally or ideologically.
Data ownership does exist in the Fediverse, in fact it is one of its selling points that you can set up your server and own the data instead of using a surveillance capitalist SaaS that stores, manipulates and imposes legal rights over your data. Applications like Mastodon do send a federation request to other instances to delete data if submitters want to. Additionally, some users put licenses on their profile that might have restrictions (i.e: CC non-commerical, etc.) on what you are legally allowed to do with the data.
So no, accessing the data is not the same as using or processing it for many people, legally too in several parts of the world. Also, "innocuous curiosities" label is entirely subjective.
I think the more commonly held belief is that the data, while unfortunately available right now, was going to become more secure in the near future; but the exodus from reddit happened too soon. So now there is a lot of that data, and the better management and protection of it hasn't had time enough to happen.
In comes you, seeing the opportunity, and you seek to exploit it.
I think this is fine - people I've seen who objected to this kind of project were more about their account being indexed. Projects like respective size of instances were always fine
Probably would need to scrap this data from all the various instances to build up the dataset:
I mean the fediverse isn't exactly super mature; from what I know you should expect that you'll have to generate that data yourself some way or another.
I don't think you could do it without DB access, but not 100% sure