TheDude

Hello sh.itjust.works community,

It’s been a while since my last update, and I owe you all a more detailed one soon. However, today I’m here to give you a heads-up about the upcoming upgrade.

Over the next 7 days, I’ll be upgrading the instance to v19.5. While this update doesn’t strictly require it, I’ve decided to take this opportunity to upgrade our Postgres database from version 15 to 16 as well. This process can be time-consuming and have take up to 2 hours to complete on other instance.

I'm optimistic that we'll complete it in less time, but to err on the side of caution, I've scheduled the maintenance window for 120 minutes.

To minimize downtime, I've split the upgrade into two phases: Phase one will cover the database update, and Phase two will handle the Lemmy code upgrade.

Note: If the database upgrade is completed in under 60 minutes, I’ll proceed with the Lemmy code upgrade during the same maintenance window.

Here's the tentative timeline:

During these two planned events those who want to provide moral support or who want to get periodic updates are more than welcome to join us on our matrix channel

edit: 9:23EDT - Hey all, the database and lemmy codebase has been upgraded to v19.5. The database upgrade took a lot less time than expected so I went right into upgrading the codebase. A whole bunch of background tasks are still running which might make your experience slower than usual. Things should be back to normal in the next couple of hours.

Hello sh.itjust.works community,

Many of you have been eager to get an update about when the sh.itjust.works instance will get it's upgrade to the latest version of lemmy. Here's a update along with a tentative timeline.

In December 2023 I purchased a new server for this community. It took me awhile but I eventually made the time to get it racked at the local datacenter. For the sysadmins lingering and those interested here are the specs:

• Dual Xeon 2.9Ghz CPUs (32 cores total)
• 256GB ram
• 4 x 1TB SSD in raid 10 (with room to add 6 more disks)
• 10gbit networking

While I'm ready to proceed with the upgrade, I've decided to first migrate this instance over to the new hardware. Here are two reasons.

1. Those of you who have been around long enough may remember that I've been running this instance on "borrowed" unused resources that were available at the time. There are no more resources available for this instance to grow.
2. There are reports that the latest version of lemmy may use more resources. Given we are among the bigger instances, should I end up in a situation where I need to increase resources to keep things fast I'll be restricted.

Here's the tentative timeline:

• If anything major goes wrong on the 27th I will revert back the changes and bring the instance back up on the current server.
• If anything major goes wrong on the 29th I will revert back using an earlier snapshot. If that fails, I will restore from a backup.

During these two planned events those who want to provide moral support or who want to get periodic updates are more than welcome to join us on our matrix channel

=========================================================
Update February 29 2024
We've successfully completed the upgrade to v1.9.3. I'm happy to announce that we did it in an astonishing 27 minutes, a whole 93 minutes under what was expected. The extra leg work that was done over the last few weeks combined with the better hardware definitely played a part. Looking over the processes, it looks like the service responsible for images is still doing some work so it's possible that you will come across some broken images. I'll be keeping on eye on that over the next bit and make adjustments if needed. Thank you all for the support and to all of you who kept me company on our matrix channel. Have a good evening.

=========================================================
Update February 27 2024
We've successfully completed the migration. I'm happy to announce that this instance is now running on its new hardware dedicated solely to this community! We experienced just under 40 minutes of downtime which is a whole 50 minutes less than expected. Please give this instance a chance to catch up what it missed but we should be good within the next 30 or so minutes. Thank you

Hey all,

Similar to the lemmy.world announcement, we have defederated from Rammy.

Want to discuss this more? I created a [Discussion] post for us to do just that.

This instance has been upgrade to 0.18.0.

The lemmy developers have removed the built in captcha so I had to enable email verification. They will be re-implementing this feature in the next update.

Lots of new improvements in this version. The Link i wanted to share is on the lemmy.ml instance and they appear to be down at the moment. Hopefully not related to this update.

Stay tuned.

Edit: Lemmy.ml back up, Here is the post: link of the update

Hey all

Today, I am thrilled to share two significant developments with you that will determine our shared future.

Firstly, the issue of donations. Since the inception of this instance, your most frequent request has been the ability to make contributions to support my initiative. While initially, I had never intended to accept donations, I've come to realize the value this brings in ensuring our platform's sustainability. In response to your requests, within the next week, I will be introducing several options for those of you who wish to donate. I want to emphasize that these donations are entirely optional and will directly support our instance's operational necessities - dedicated hardware, colocation fees, email services, and more.

The second announcement needs a full disclosure: it could be an extraordinary idea or potentially not so. Since the beginning of this instance, I've seen a number of insightful posts, recommendations, and ideas that the community has put forward, often superior to what I could have conceptualized myself. Even our instance's name was born from one of your suggestions.

Frequently, I'm approached by users seeking clarity on our rules and guidelines or expressing their thoughts on existing rules. In reflecting upon this, it became clear to me that I've been attempting to determine what's best for the community. But who am I to make these decisions? Just two weeks ago, I was a user among you all. Hence, I'm moving away from the traditional role of decision-making.

Instead, I want to hand over the reins of decision-making to you - the community. I'm excited to announce the creation of a community called agora where you can express your desires for the future of this instance. It's up to you to come together, discuss, and reach a consensus. If you wish to add, remove, or modify a rule, make a post, garner support from other members, and I'll implement the change. This invitation extends beyond our immediate community - I welcome input from everyone across the fediverse. Again to be clear, I gave an example of modifying rules but this applies to anything that I have the ability to do on this instance.

As the instance owner, I only have one caveat to add - any decisions made should not jeopardize the instance's existence or result in legal complications. Aside from that, I'm eager to see where this new direction takes us.

Thank you for your ongoing support, creativity, and engagement.

Like many of you, I woke up this morning to discover that our instance, along with lemmy.world, had been unexpectedly added to the beehaw block list. Although this development initially caught me off guard, the administrators at beehaw made an announcement shedding light on their decision.

The primary concern raised was our instance's policy of open registration. Given my belief that the fediverse is still navigating its early stages, I believe that for it to mature, gain traction, and encourage adoption, it is crucial for instances to offer an uncomplicated and direct route for newcomers to join and participate. This was one of the reason I decided to launch this instance. However, I do acknowledge that this inclusive approach brings its unique challenges, including the potential for toxicity and trolls. Despite these hurdles, I maintain the conviction that our collective strength as a community can overcome these issues.

After this happened, the beehaw admins and I had a good chat about their decision. While our stances on registration policies might diverge, we realized that our ultimate goals are aligned: we both strive to foster communities that thrive in an atmosphere of safety and respect, where users can passionately engage in discussions and feel a sense of belonging.

Although the probability of an immediate reversal are slim given the current circumstances, I believe we have managed to identify common ground. It's evident that, even in separation, we can unite to contribute positively to the broader fediverse community.

In the coming weeks or months, we plan to collaborate with other lemmy instance administrators to suggest enhancements and modifications to the lemmy project. Primarily, our proposals will concentrate on devising tools and features that empower us, as instance administrators, to moderate our platforms effectively.

In the meantime, while I understand may not be ideal for everyone, users who choose to participate on the beehaw instance will be required to register a separate account on their instance.

Thank you all for continuing to make this community great!

Good morning,

Some of you may have experienced a small hiccup this morning (eastern time). I went ahead and upgraded our lemmy instance to 0.17.4. I also took this opportunity to make some small adjustments to the instance that required a restart of services. These changes were mainly related to increasing Federation worker count in hopes to decrease synchronization times with other instances. A small fix to help slow down the storage growth was also applied (limiting log sizes that get generated).

Everything should be back up now. In the future my goal is to be more transparent with scheduling downtime.

edit: updated link

I'm aware that some of you have been getting some errors loading this instance. This was a configuration that I needed to be adjusted which has since been.

Do be patient if we run into other issues, I'll be continuously working in the back-end with others to better your overall experience.

Still a lot of capacity available!

edit: For those that are interested in the configuring changes. I was using the default lemmy configuration for the nginx worker_connections value. This value needed to be raised. Shout out to @ruud@lemmy.world over at lemmy.world for helping me out.

Hey all, I created this community thinking I would have time to mod and post in it but its pretty clear now that this won't be the case. If anyone is interested in taking over this community and has the time to properly moderate it please let me know!

Hello everyone!

I'd like to first start by welcoming all the new users who have decided to chose sh.itjust.works as their lemmy home instance. It's wild to think that its only been 4 days since I've decided to spin up this instance and we are already nearing 1000 registered users.

During these last few days I've welcomed users from all parts of the world who collectively have created nearly 50 new communities on this instance alone.

Today I want to be transparent and share some of my thoughts, clarify some rules, share some concerns, answer some questions and most importantly introduce the two new instance admins that have been added over the last couple of days. So let's get started!

New Admins

• @smorks@lemmy.ca - Some of you may recognized this name from the lemmy.ca instance. This is in fact the same person. I've asked smorks to help out on this instance and he accepted. As the userbase grows having someone whos living some of the similar issues will be bring great value to this instance. We are stronger together!
• @donut@sh.itjust.works - donut is a personal friend, new to the fediverse but has agreed to give me a hand should things get out of control. He's a little quiet right now but he'll open up soon enough.

Rules

As you may have noticed, there aren't a whole lot of rules. This might need to change in the future but my goal as of right now is to keep it this way. Being respectful to all users no matter which instance they come from, part of the world they grew up in, personal believe or personal opinion is a absolute must. Our world is already in shambles, we don't need another place for all of us to fight against one another.

• Be respectful
• No bigotry
• No Ads or Spamming
• No pornography

You'll may have noticed that a new rule that has been added. No pornography. When pornography is introduced a whole bunch of legal implications need to be considered amongst other things like proper moderation, DMCA notices processes, etc. 4 days ago I would not have thought this to be a problem however I am not in a position to hire legal counsel and implement the elements required to not get this instance shutdown. So for now, no pornography. If the demand grows for this in the future and there is an opportunity to fill this gap I promise I will revisit this rule.

Donations

I've had a few users ask me whether they could start donating. I'll admit I was caught off guard when that happened, thinking of how thoughtful people can be. When I started this instance 4 days ago, I did it because I wanted people to have a home away from reddit, a place where they could come and share interesting topics, debate, discuss and connect.

I'm by no means rich but I have chosen a career that puts me in proximity of things needed to run an instance like this one. It also does help that I really like servers and while some people might really like the latest limited edition sneakers from X company or the latest iPhone/Android device, etc..., I really really like servers, datacenters and networking.

I don't have a plan for accepting donations today but seeing how this community is growing, I will need to consider it soon. You know what's more expensive than running a lemmy instances for thousands of users? a failed marriage.

There are some other options that might be available such as looking for sponsors but I do want to keep this instance about the users and not about corporate so I'll need to think hard about this one.

Whatever is decided I'll make another post with the details.

Concerns

Let's quickly talk about concerns. Most of you already know that many of the reddit mods are orchestrating a protest by blacking out many popular subreddits. Most of us anticipate a large spike in traffic towards our instances during the period of June 12 to 14th. If any of you were around during the Digg days, you may remember how the migration from digg to Reddit went. The early reddit days was filled with outages, performance issues, and frustration.

A lot of people people are looking for what might be the next reddit. We don't knows if lemmy will be it but a lot of people will surely be giving it a shot. During that time many instances including this one will likely deal with some growing pains.

From a resources perspective my biggest worry for this instance is related to the storage. I have CPUs, I have RAM but when it comes to storage, well its an element that has been growing much faster than I had anticipated. I will be keeping a close eye on this however want to wave a little yellow flag so that everyone is aware until I work out a solution.

Q&A

What communities allowed to be homed here?

• As long as it follows the rules above and doesn't get any DMCA requests or jeopardize this instance from being shutdown or me in trouble. Go nuts!

What communities allowed to be federated over/subscribed to?

• Same as the above and follow rules.

What am I allowed to link to in communities or text posts?

• Same as the above and follow rules.

What kind of images am I allowed to upload to the integrated image host?

• Storage is something that will likely become a problem in the foreseeable future. You all upload a lot! That being said, don't upload copyrighted materials or anything that will get me in trouble!

A little story that goes along with this image. I asked my wife to give me a prompt of anything to have an AI generate an image. This is what she came up with:

A desert taco with ice cream and chocolate drizzle and topped with sprinkles next to a desert burrito filled with bananas and peanut butter topped with strawberries and English cream

She has since left the home and I fear I might have to pick her up at the local desert shop.

As the reddit mods gets ready for the June 12-14 black-out, there some anticipation that an influx in user base will shift over to many of the lemmy instances as user seek out a home to post their internet memes and discuss their interests.

In anticipation of this increased volume I will be growing our current instance from

• 16 CPU
• 8 GB ram

to

• 24 CPU
• 64 GB ram

This server is currently equipped with SSDs that are configured in a raid 10 array (NVMEs will come in the next gen that get deployed)

Earlier today I also configured some monitoring that I'll be watching closely in order to have a better understanding on how the lemmy platform does under stress (for science!)

I'll be sharing graphs and some other insights in this thread for everyone that is interested. Feel free to ask anything you might be interested in knowing more of!