this post was submitted on 16 Jun 2023
42 points (100.0% liked)
Asklemmy
43851 readers
1362 users here now
A loosely moderated place to ask open-ended questions
Search asklemmy ๐
If your post meets the following criteria, it's welcome here!
- Open-ended question
- Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
- Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
- Not ad nauseam inducing: please make sure it is a question that would be new to most members
- An actual topic of discussion
Looking for support?
Looking for a community?
- Lemmyverse: community search
- sub.rehab: maps old subreddits to fediverse options, marks official as such
- !lemmy411@lemmy.ca: a community for finding communities
~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
https://lemmy.ml/robots.txt , https://lemmy.world/robots.txt , etc don't seem to disallow posts, so the text-based content should be easy to index, at least for these instances.
related news: Google is getting a lot worse because of the Reddit blackouts.
An earlier post pointed out: federated sites seem like they will suffer against central content in a SEO world - regardless of whether they are technically indexable.
I wonder if lemmy should have a SEO friendly federated site... .com domain, robots.txt and everything else...
right, though the SEO game is changing drastically with AI. People are using GPT-like models often in place of searches and likewise, expecting search results to hit their answers rather than being vague pointers. Following this reasoning, the search engines will need direct users to where the valuable information is, not always, but often enough to not lose users to competitors.
So the thing about SEO is that it's often an attention game that advertisers and smaller websites compete with each other. The information in public forums and threads is invaluable for the success of the search engine itself, so they're the ones that will eventually have to adapt to the new federated reality, should it become mainstream - and I do hope so.
How long does it usually take for google to index websites? Because I tried the string
lemmy site:lemmy.ml after:2023-06-15and only one post turned up for me and it wasMemes... the current state of affairs does not seem promising ๐ And if I tried with another instance with the same keywordslemmy site:kbin.social after:2023-06-15nothing even turned up.I wonder though, will search engines adapt to Lemmy and its fediverse system? Or will search engines die? Or will we see dedicated search engines to search through the fediverse?
Anything between a couple of hours to more than a week, I don't think having a "real-time feed" through Google is important though. Other than world cup scores, their results were never about speed.