this post was submitted on 01 Jul 2023
3683 points (97.1% liked)

Lemmy.World Announcements

29079 readers
223 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world/

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

Donations πŸ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 2 years ago
MODERATORS
 

Looks like it works.

Edit still see some performance issues. Needs more troubleshooting

Update: Registrations re-opened We encountered a bug where people could not log in, see https://github.com/LemmyNet/lemmy/issues/3422#issuecomment-1616112264 . As a workaround we opened registrations.

Thanks

First of all, I would like to thank the Lemmy.world team and the 2 admins of other servers @stanford@discuss.as200950.com and @sunaurus@lemm.ee for their help! We did some thorough troubleshooting to get this working!

The upgrade

The upgrade itself isn't too hard. Create a backup, and then change the image names in the docker-compose.yml and restart.

But, like the first 2 tries, after a few minutes the site started getting slow until it stopped responding. Then the troubleshooting started.

The solutions

What I had noticed previously, is that the lemmy container could reach around 1500% CPU usage, above that the site got slow. Which is weird, because the server has 64 threads, so 6400% should be the max. So we tried what @sunaurus@lemm.ee had suggested before: we created extra lemmy containers to spread the load. (And extra lemmy-ui containers). And used nginx to load balance between them.

Et voilΓ . That seems to work.

Also, as suggested by him, we start the lemmy containers with the scheduler disabled, and have 1 extra lemmy running with the scheduler enabled, unused for other stuff.

There will be room for improvement, and probably new bugs, but we're very happy lemmy.world is now at 0.18.1-rc. This fixes a lot of bugs.

(page 7) 50 comments
sorted by: hot top controversial new old
[–] Gort@lemmy.world 2 points 1 year ago

Thanks for being both informative and open about this issue. Hope it works out.

Edit: The first two versions of this post didn't seem to send: spinning post button. Maybe it's not fixed or it's struggling?

[–] free@lemmy.world 2 points 1 year ago

U peeps are awesome. πŸ™πŸ™‡

[–] thaedrus@lemmy.world 2 points 1 year ago

Looks like everything is working well so far. Browsing from Wefwef and Liftoff.

Thank you!

[–] BitOneZero@lemmy.world 2 points 1 year ago (1 children)

we created extra lemmy containers to spread the load. (And extra lemmy-ui containers).

Is Rust HTTP server running into thread limits? database connection pooling? All kinds of internal questions bout that solution.

load more comments (1 replies)
[–] dukethorion@lemmy.world 2 points 1 year ago (1 children)

Can't edit profile settings / info. Pushing Save does nothing.

load more comments (1 replies)
[–] GwilliamSP@lemmy.world 2 points 1 year ago

Amazing! Thank you ruud and everyone who helped you push through it.

[–] smokedclover@lemmy.world 2 points 1 year ago

Yay, i could finally login. Thank you for the great work and thank you for keeping us updated!

[–] Sterben@lemmy.world 2 points 1 year ago

Congratulations! Works perfectly for me, thank you! πŸ€™

[–] CataclysmZA@lemmy.world 2 points 1 year ago

Posting this from Jerboa. I am grateful for the hard work!

[–] FinalBoy1975@lemmy.world 2 points 1 year ago (2 children)

You know, this is a nice post because now I understand what was happening to me as a user. Thanks for confirming that I am not insane! Well, maybe I'm insane, but what I was trying to do and couldn't was real, not something I was doing wrong. Also, thanks for updating the stuff that makes it work.

load more comments (2 replies)
[–] Teal@lemmy.world 2 points 1 year ago

I was having trouble earlier but now able to log in just fine on browser. Voting on posts doesn't seem to be working for me on desktop or apps. In apps I keep seeing error notices about votes not going through and desktop browser (Firefox) doesn't work but there's no notification there. Anyone else? Maybe everything needs a little time to sync up.

Thank you so much!

[–] spasm01@lemmy.world 2 points 1 year ago (4 children)

I cannot see my subscriptions, not sure if its me or what. On Jerboa 0.34

load more comments (4 replies)
[–] oldgreg@lemmy.world 2 points 1 year ago
[–] root@lemmy.world 2 points 1 year ago

Great work, and thank you for the thorough updates?

[–] thisisdee@lemmy.world 2 points 1 year ago

Thanks for all your work! Love the new look and it’s working great now

[–] Anaralah_Belore223@lemmy.world 2 points 1 year ago* (last edited 1 year ago) (2 children)

Comment testing on lemmy.world

Why posting/commenting on other communities = blazing fast, while here = takes about 3 to 8 seconds? Is it just me?

load more comments (2 replies)
[–] MrPommeroy@lemmy.world 2 points 1 year ago

Thanks for all the hard work and documentation!

[–] solongaphasia@lemmy.world 2 points 1 year ago

thank you for all you’re doing on this. godspeed

Does browsing with Incognito/Private mode opens up new bugs, or does the refreshing thing kept the same principle? I should be stayed as logged on, but for some reason - after this update - whenever I open a new private tab from the tab I'm logged on I am indicated as not logged in.

[–] Sigmatics@lemmy.world 2 points 1 year ago

Still seeing a lot of 504s, and this is at a time where the site shouldn't be super busy

[–] bappity@lemmy.world 2 points 1 year ago

thank you for upgrading! I much prefer the new URL structure of this version

[–] Dude1324@lemmy.world 2 points 1 year ago

❀️

[–] AlmightySnoo@lemmy.world 2 points 1 year ago

lemmy.world finally fast as fuck and the login form seems to work again, thanks for all your hard work!

load more comments
view more: β€Ή prev next β€Ί