edit: This start bit is wrong; Lemmy does SSR so Javascript-free/spiders should see at least some comments.
~~Lemmy is currently pretty terrible at SEO, in large part because the comments don't load until the JS has run.~~
~~This isn't just a problem for search engines, it affect things like archive.org and offline reading. Earlier today I loaded a page from an instance that had dropped offline - while they had Cloudflare Always Online enabled, the page loaded without comments so it was almost useless.~~
I think it's a mistake to consider all the SEO-related concerns as irrelevant just because you don't care about Google, etc. Most of the things necessary for good SEO are just good practices, with benefits for all users, especially in the areas of accessibility and third-party tools.
Hmm, I've just tested an that does seem to be the case, at least as far back as 0.17.4. Do you know when this was added? Or if it's something that can be disabled?
Looking into Cloudflare Always Online, it uses the Internet Archive's backup instead of keeping on itself which could explain me seeing zero comments (i.e. IA scraped the page after posting but before any comments). I can't figure out which page in my history was the post in question, so I can't be sure.