I saw this a while ago when I was searching high and low for a messenger to switch to from Element (didn't end up switching to anything). I like a lot of things about it, but don't like that it can't support offline delivery or multi-device. I think no messenger without those features will ever take off because too many people demand them. In fact I remember that in group chats, it won't even do the faux offline messaging that it does for 1-to-1 chats! I don't know why but that was the biggest deal breaker for my own use case. I hope they eventually fix that, there's no reason they can't.
I also don't like that it has moderation features, which feel really out of place to me in a peer to peer system. I don't want that kind of social hierarchy in a context that more or less inherently assumes all participants are trusted. Moderation only makes sense in public forums IMO.
There's also at least one security cocern: https://github.com/TokTok/c-toxcore/issues/426 Though the people pointing it out were pretty toxic and really exaggerated its severity :(
Despite all this, if offline messaging and multi-device aren't requirements for you, I'd say Tox is a great option, maybe the best option.
I'm proud to hear you're getting into Gemini, by the way :) I first heard of it a while ago but recently got really onboard and I'm working on converting my own website to Gemini.