Fedia Discussions

1 readers

5 users here now

founded 1 year ago

MODERATORS

jerry@fedia.io

Fedia update (fedia.io)

submitted 7 months ago by jerry@fedia.io to c/fedia@fedia.io

10 comments fedilink hide all child comments

You have no doubt noticed that federation is breaking again. I am painfully aware of it. The issue is with the symphony queue runner that processes incoming messages from other instances. Occasionally, the server receives a message that causes the queue runner to die. I have to manually remove the offending message out of rabbitmq. The message does not appear to be malicious, rather there is something malformed in an otherwise legit looking post that causes the queue to die. I am working with the mbin team to track down what it is about the messages that causes the problem, but sadly until I there is a fix, this is going to keep happening

top 10 comments

sorted by: hot top controversial new old

[–] Nougat@fedia.io 6 points 7 months ago (1 children)

Growing pains are to be expected. You're probably aware that some people (myself included) are shifting here [from|in addition to] kbin.social; that extra load probably doesn't help.

[–] jerry@fedia.io 1 points 7 months ago (2 children)

Ah - that is what we’re here for. I know kbin has had a cloud of uncertainty around it. Did something recently happen on kbin.social?

[–] Nougat@fedia.io 3 points 7 months ago

Ernest made a post today, yes, but kbin.social has reached a point which demands a next level of administration (from both technical and non-technical perspectives). While I want that project to thrive, there is writing on the wall which unfortunately cannot be ignored.

[–] Rhaedas@fedia.io 2 points 7 months ago

Ernest's reply today to questions about his absence. Kbin hasn't been abandoned, just life getting in the way, with hope that it will improve shortly.

[–] jerry@fedia.io 4 points 7 months ago (1 children)

The good news is that I think I figured out where the problematic messages are coming from. Now I have to figure out what it is about them.

[–] Nougat@fedia.io 1 points 6 months ago (1 children)

Seems to be roughly 1.7 million times better today.

[–] jerry@fedia.io 1 points 6 months ago

it took 3 days to process the backlog, but it's caught up now and I've not seen any re-occurrence of the prior problem.

[–] Hypx@fedia.io 4 points 6 months ago

Some things seem to be fixed. But I'm stilling noticing that many communities are not reachable. I mentioned about them here: https://fedia.io/m/fedia/t/590616/-/comment/3994532

[–] jerry@fedia.io 4 points 7 months ago (1 children)

The server is busily processing the 1,200,000 messages that queued up over the past 20 hours. It’s died 3 times in the past few minutes, so I’m not optimistic about how long this will take

[–] jerry@fedia.io 2 points 7 months ago

Up to 1,700,000 in the queue 😱