My problem with Lemmy is the lack of activity in niche communities. You're right that there needs to be a critical mass and arguably Lemmy has it, but only for the most mainstream, generic type of content. It doesn't have the mass to sustain any sort of niche, outside of maybe tech related topics because of the way the userbase is slanted.
I find myself going back there often because of that, but I hope that the userbase for generic content enough to sustain and grow, from where more active niche communities can spring up.