this post was submitted on 01 Jul 2023
3683 points (97.1% liked)

Lemmy.World Announcements

29022 readers
8 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news ๐Ÿ˜

Outages ๐Ÿ”ฅ

https://status.lemmy.world

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

Donations ๐Ÿ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 1 year ago
MODERATORS
 

Looks like it works.

Edit still see some performance issues. Needs more troubleshooting

Update: Registrations re-opened We encountered a bug where people could not log in, see https://github.com/LemmyNet/lemmy/issues/3422#issuecomment-1616112264 . As a workaround we opened registrations.

Thanks

First of all, I would like to thank the Lemmy.world team and the 2 admins of other servers @stanford@discuss.as200950.com and @sunaurus@lemm.ee for their help! We did some thorough troubleshooting to get this working!

The upgrade

The upgrade itself isn't too hard. Create a backup, and then change the image names in the docker-compose.yml and restart.

But, like the first 2 tries, after a few minutes the site started getting slow until it stopped responding. Then the troubleshooting started.

The solutions

What I had noticed previously, is that the lemmy container could reach around 1500% CPU usage, above that the site got slow. Which is weird, because the server has 64 threads, so 6400% should be the max. So we tried what @sunaurus@lemm.ee had suggested before: we created extra lemmy containers to spread the load. (And extra lemmy-ui containers). And used nginx to load balance between them.

Et voilร . That seems to work.

Also, as suggested by him, we start the lemmy containers with the scheduler disabled, and have 1 extra lemmy running with the scheduler enabled, unused for other stuff.

There will be room for improvement, and probably new bugs, but we're very happy lemmy.world is now at 0.18.1-rc. This fixes a lot of bugs.

you are viewing a single comment's thread
view the rest of the comments
[โ€“] Sneckster@lemmy.world 172 points 1 year ago (4 children)

This post encapsulates what I love about this place.

[โ€“] jugalator@lemmy.world 101 points 1 year ago (1 children)

It's almost like we're talking to a normal human being rather than an evil overlord.

[โ€“] lordbarbarossa@lemmy.world 32 points 1 year ago (2 children)

I for one welcome our new benevolent overlords

[โ€“] muffedtrims@lemmy.world 5 points 1 year ago

Don't blame me, I voted for Kodos

[โ€“] deweydecibel@lemmy.world 59 points 1 year ago (1 children)

How so?

Admin communicating information about the site to users like genuine human beings, instead of the corporate-sanitized pablum reddit admins speak in? Yeah that's refreshing.

[โ€“] Baylahoo@lemmy.world 25 points 1 year ago

It shouldn't be unreasonable but it is damn refreshing.

[โ€“] kaseijin@lemmy.world 6 points 1 year ago

It's cool we all want the new community to succeed and get live updates on things being fixed or worked on. There's some shared misery in growing pains/bugs as we all stress test the system, but I think that only somehow brings more communal joy when the problems are fixed.

Agreed. Thanks for letting us nerd out