lemmyvore

joined 1 year ago
[–] lemmyvore@feddit.nl 4 points 1 day ago

If you end up with resizing /var as the only solution, please post your partition layout first and ask, don't rush into it. A screenshot from an app like Disk Manager or Gparted should do it, and we'll explain the steps and the risks.

When you're ready to resize, you MUST use a bootable stick, not resize from inside the running system. You have to make a stick using something like Ventoy, and drop the ISO for the live version of GParted on the stick, then boot with it and pick the Gparted live. You'll have to write down the instructions and be careful what you do, and also hope that there's no power outage during.

[–] lemmyvore@feddit.nl 4 points 1 day ago (1 children)

The safest method, if your /home has enough space, is to use it instead of /var for (some) Flatpak installs. You can force any Flatpak install to go to /home by adding --user to the command.

If you look at the output of flatpak list it will tell you which package is installed in user home dir and which in system (/var). You can also show the size of each package with flatpak list --columns=name,application,version,size,installation.

I don't think you can move installed apps directly between system/user like Steam can (Flatpak is REALLY overdue for a good package manager) but you can uninstall apps from system, then run flatpak remove --unused, then install them again with --user.

Please note that apps installed with --user are only seen by the user that installed them. Also you'll have to cleanup separately for system and user(s) in the future (flatpak remove --unused for system, then flatpak remove --unused --user for each user).

[–] lemmyvore@feddit.nl 1 points 1 day ago

It's not an issue on Arch & derivates, due to the simple fact I mentioned above: third-party (AUR) packages are never allowed to use the name of an official package.

If a third-party package was already using a name that a new official package wishes to use, users are required to willingly uninstall the third-party package in order to be allowed to install the official one, and can never re-install the third-party package unless it changes its name.

It also helps that there's only one third-party repo (the AUR) so it prevents name overlaps among third-party packages. Although that's of secondary importance since it can be bypassed by crafting custom packages locally.

I appreciate the difficulty of enacting such a rule on Debian or Ubuntu now, considering the vast amount of already existing, widely established third-party repos, and also the fact that Debian official repos contain 3-4 times as many packages as Arch official repos. Which is why I think there's no way to fix this aspect of Debian/Ubuntu anymore.

I'm not saying that makes them unusable... but I believe that anybody who uses them should be [made] aware of this caveat. It's not readily apparent and by the time it bites a new user she's probably already invested a couple of years in them.

[–] lemmyvore@feddit.nl 1 points 2 days ago (2 children)

Interesting, I'll keep it in mind.

Still not sure it would help in all cases. Particularly when 3rd party repos have to override core packages because they need to be patched to support whatever they're installing. Which is another very bad practice in the Ubuntu/Debian world, granted.

[–] lemmyvore@feddit.nl 1 points 2 days ago (4 children)

I'm not sure how that would help. First of all, it would still end up blocking proper updates. Secondly, it's hard to figure out what exactly you're supposed to pin.

[–] lemmyvore@feddit.nl 5 points 3 days ago (6 children)

Third party package mechanism is fundamentally broken in Ubuntu (and in Debian).

Third party repos should never be allowed to use package names from the core repos. But they are, so they pretend they're core packages, but use different version names, and at upgrade time the updater doesn't know what to do with those version and how to solve dependencies.

That leaves you with a broken system where you can't upgrade and can't do anything entirely l eventually except a clean reinstall.

After this happened several times while using Ubuntu I resorted to leaving more and more time between major upgrades, running old versions on extended support or even unsupported.

Eventually I figured that if I'm gonna reinstall from scratch I might as well install a different distro.

I should note I still run Debian on my server, because that's a basic install with just core packages and everything else runs in Docker.

So if you delegate your package management to a completely different tool, like Flatpak, I guess you can continue to use Ubuntu. But it seems dumb to be required to resort to Flatpak to make Ubuntu usable.

[–] lemmyvore@feddit.nl 2 points 3 days ago (2 children)

How do you avoid interaction if it's being done automatically by your machine when you open up a print dialog, and if malicious servers can use the same names as legit printers?

[–] lemmyvore@feddit.nl 2 points 4 days ago* (last edited 4 days ago)

People often think that things like recording your screen or keylogging are the worst but they're not. These attacks would require you to be targeted by someone looking for something specific.

Meanwhile automated attacks can copy all your files, or encrypt them (ransomware), search for sensitive information, or use your hardware for bad things (crypto mining, spam, DDoS, spreading the malware further), or most likely all of the above.

Automated attacks are much more dangerous and pervasive because they are conducted at massive scale. Bots scan massive amounts of IPs and try all the known exploits and vulnerabilities without getting tired, without caring how daunting it may be, without even caring if they're trying the right vulnerability against the right kind of OS or app. They just spray everything and see what sticks.

You're thousands of times more likely to be caught by such malware than it is to be targeted by someone with the skill and motive to record your screen or your keyboard.

Secondly, if someone like that targets you and has access to your user account, Wayland won't stop them. They can gain access to your root account, they can install elevated spyware, they can patch Wayland and so on.

What Wayland is doing is the equivalent of asking you to wear a motorcycle helmet 24/7, just in case you slip on some spilled juice, or a flower pot falls on your head, or the bus you're in crashes. All those things are possible and the helmet would come in handy but are they likely? We don't do it because it's not, and it would be a major inconvenience.

[–] lemmyvore@feddit.nl 0 points 5 days ago

You were merely lucky that they didn't break.

Lucky... over 5 years and with a hundred AUR packages installed at any given time? I should play the lottery.

I've noticed you haven't given me any example of AUR packages that can't be installed on Manjaro right now, btw.

it wasn't just a rise in popularity of Arch it was Manjaro's PAMAC sending too many requests DDoSing the AUR.

You do realize that was never conlusively established, right? (1) Manjaro was already using search caching when that occured so they had no way to spam AUR, (2) there's more than one distro using pamac, and (3) anybody can use "pamac" as a user agent and there's no way to tell if it's coming from an actual Manjaro install.

My money is on someone actually DDoS'ing AUR and using pamac as a convenient scapegoat.

Last but not least you're trying to use this to divert from the fact AUR packages work fine on Manjaro.

[–] lemmyvore@feddit.nl -1 points 5 days ago

That's exactly the problem. Wayland is a set of standards, more akin to FreeDesktop.Org than to X. It lives and dies by its implementations, and it's so utterly dependent on them that "KDE Wayland" has started to become its own thing. KDE are pretty much forging ahead alone nowadays and when they make changes it becomes the way to do it. Also what they do can't be shared with other desktops because they'd have to use KDE's own subsystems and become dependent on its whims.

It wasn't supposed to be "Kdeland" and "Gnomeland" but that's what it's slowly becoming. We're looking at major fragmentation of the Linux desktop because desktop teams have and do stop seeing eye to eye on major issues all the time. And because there's no central implementation to keep them working together they're free to do their own thing.

[–] lemmyvore@feddit.nl 4 points 5 days ago (2 children)

We need to keep a balance between security and convenience, to avoid systems becoming too awkward to use. Wayland tipped this balance too far on the side of security. Malicious local exploitation of the graphics stack has never been a big issue; consider the fact that someone or something would need to compromise your own account locally, at which point they could do much worse things than moving your windows around. It's not that the security threat doesn't exist, it's that Wayland has approached it at the wrong end and killed a lot of useful functionality in the process.

Also consider that this issue has existed for the entire history of desktop graphics on *nix and nobody has ever deemed it worth to destroy automation for it. If it were such a grave security hole surely someone would have raised the alarm and fixed it during all this time.

My opinion is that Wayland has been using this as a red herring, to bolster its value proposition.

[–] lemmyvore@feddit.nl 0 points 5 days ago (2 children)

Manjaro has no purpose, it's half-assed at being arch and it's half-assed at being stable.

My experience with Manjaro and Fedora, OpenSUSE etc. contradicts yours. Manjaro has the best balance between stability and rolling out of the box I've seen.

"Out of the box" is key here. You can tweak any distro into doing anything you want, given enough time and effort. Manjaro achieves a good balance without the user having to do anything. I remind you that I've tested this with non-experienced users and they have no problem using it without any admin skills (or any admin access).

Debian testing is a rolling.

It is not.

AUR isn't a problem in Manjaro because of lack of support, it's a problem because packages there are made with Arch and 99.999% of its derivatives in mind, aka latest packages not one week old still-broken packages.

And yet I've managed to install dozens of AUR packages just fine. How do you explain that?

Matter of fact, I've never run into an AUR package I couldn't install on Manjaro. What package is giving you trouble?

Manjaro literally accidentally DDoSes the AUR every now and then because again they're incompetent.

You're being confused.

AUR had very little bandwidth to begin with and could not cope with the rise in popularity of Arch-based distros. That's a problem that needs to be solved by the AUR repo first and foremost. Manjaro did what they could when the problem became apparent and has added caching wherever it could. Both Manjaro and Arch devs have worked together to improve this.

 

It doesn't seem to be doing anything for me, even on large websites like YouTube or Amazon, it basically just copies the link as-is.

 

I took some photos at an event and I need to go through them and get rid of the bad ones (eyes closed, things in the shot, out of focus, blurred etc.) I'm not a pro photographer so no idea where to begin with photo apps. I've used RawTherapee and Gimp a bit.

What app will let me quickly browse the photos and handle (delete/tag) photo formats together (both the RAW and the JPG)?

 

I'm posting this in selfhosted because Gandi increasing prices actually helped me a lot with being more serious about selfhosting, made me look into things like DNS and reverse proxies and VPN and docker and also ended up saving me money by re-evaluating my service needs.

For background, Gandi.net is a large and old (25 years) domain registrar and hosting provider in the EU, who after two successive rounds of being acquired by investment funds have hiked up prices across the board for all their services.

In July 2023 when they announced the changes for November I was using their services for pretty much everything because I manage domains for friends and family. That means a wide selection of domains registered with them (both TLDs and European ccTLDs), LAMP hosting, and was taking advantage of their free email hosting for multiple domains.

For the record I don't hold the price hike against them, it was just unsustainable for us. Their email prices (~5€/mailbox/mo) are in line with market prices and so are hosting prices. Their domain prices are however exaggerated (€25-30/yr is their lower price now). I also think they could've been smarter about email, they could've offered lower prices if you keep domains registered with them. [These prices include the VAT for my country btw. They will appear lower in USD.]

What I did:

Domains: looked into alternative registrars with decent prices, support for all the ccTLDs I needed, DNSSEC, enforced whois privacy, and representative services (some ccTLDs require a local contact). Went with INWX.com (Germany) and Netim.com (France). Saved about €70/yr. Could have saved more for .org/.net/.com domains with an American registrar but didn't want to spread too thin.

DNS: learned to use a dedicated DNS service, especially now that I was using multiple registrars since I didn't want to manage DNS in multiple places. Wanted something with support for DNSSEC and API. Went with deSEC.io (Germany) as main service and Bunny.net (Slovenia) as backup. deSEC is free, more on Bunny pricing below. Learned a lot about DNS in the process.

Email: having multiple low-volume mailboxes forced me to look into volume-based providers who charge for storage and emails sent/received not mailboxes. I've found Migadu (Swiss with servers in France at OVH), MXRoute (self-hosted in Texas) and PurelyMail (don't know). Fair warning, they're all 1-2 man operations. But their prices are amazing because you pay a flat fee per year and can have any number of domains and mailboxes instead of monthly fees for one mailbox at one domain. Saved €130/yr. Learned a lot about MX records and SPF/DKIM/DMARC.

Hosting: had a revelation that none of the webpages I was hosting actually needed live dynamic services (like PHP and MySQL). Those that were using a CMS like WordPress or PHP photo galleries could be self-hosted in docker containers because only one person was using each, and the static output hosted on a CDN. Enter Bunny.net, who also offer CDN and static storage services. For Europe and North America it costs 1 cent per GB with a $1 minimum/mo, so basically $12/yr since all websites are low traffic personal websites. Saved another €130/yr. Learned a lot about Docker, reverse proxies and self-hosting in general.

Keep in mind that I already had a decent PC for self-hosting, but at €330 saved per year I could've afforded buying a decent machine and some storage either way.

I think separating registrars, DNS, email and hosting was a good decision because it allows a lot of flexibility should any of them have any issues, price hikes etc.

It does complicate things if I should kick the bucket – compared to having everything in one place – which is something I'll have to consider. I've put together written details for now.

Any comments or questions are welcome. If there are others that have gone through similar migrations I'd be curious what you chose.

 

I'm thinking of putting all my email archive (55k messages, about 6 GB) on a private IMAP server but I'm wondering how to access it remotely when needed.

Obviously I'd need a webmail client but is there any that can deal with that amount of data and also be able to search through To, From, Subject and body efficiently?

I can also set up a standalone search engine of some sort (the messages are stored one per file in regular folders) but then how do I view the message once I locate it?

I can also expose the IMAP server itself and see if I can find a mobile app that fits the bill but I'd rather not do that. A webmail client would be much easier to reverse proxy and protect.

 

I've repurposed a 32 GB M.2 SATA SSD as a bootable "USB stick" and I'm putting useful tools on it. So far I've got memtest, seatools, gparted live, system rescue, clonezilla, and a live install iso of the distro installed on my PC. What other great bootable tools am I sleeping on?

 

I use multiple workspaces and I open text files all the time.

Once upon a time Mousepad used to behave sanely and would open them in a new tab if there was already an instance on the current workspace, or open a new window (on the current workspace) if there wasn't.

They broke that at some point. Now it's anybody's guess where the file will open. Maybe it opens in a tab in an existing window on this workspace. Maybe in a tab in a window on a random workspace. Maybe a new window on this workspace even though there's one open. I've given up trying to figure it out.

As a last resort I can use wmctrl to figure out how to open the files and can script a sane launcher myself – provided that the editor has --tab and --window options AND lets you specify the window instance. Mousepad has the former but not the latter.

So, do you know any editor that can do it by itself or has those options so I can do it myself? TIA

 

Hi, I'm trying to find the subtitles for Harmy's "Despecialized" Star Wars remakes and I was wondering if anybody has any ideas. The original website for Project Threepio points at a blog that seems abandoned and an old private tracker (MySpleen) that never opens to public anymore. Even just the English subs would be great (the original pack contained extensive language coverage in DVD format so I was given to understand it was quite large). TIA for any hints.

 

Upgrading a self-hosted server (3)

A short intro to Docker

Docker is a lot less complicated than it was made out to be.

Docker is a way of taking a service (something like Plex) and making it work in a sort of "slice" cut out of the real machine's resources (CPU, RAM and disk space). These slices are called containers.

There are several benefits:

  • If someone breaks into one of your services, they only reach one container not the real machine, and not any of the other containers.
  • It's very easy to restore a container in case of machine reinstall, using "magical" recipe files called "docker compose yaml". If the main OS blows up you just need to reinstall stock Debian stable and Docker, then use the magical recipes.
  • The containers share similar files among themselves, so if you have 10 containers that use the same files it will only be stored once, not 10 times.
  • You can try out any server software without worrying you'll mess up your host machine. Or you can use a second configurations of the same service in a second container without worrying you'll mess up the first one.

Basic Docker commands

  • docker-compose up -d: run this into the same dir as a magical yaml file to create a container for the first time.
  • docker stop cups, docker start cups, docker restart cups will stop/start/restart the cups container.
  • docker container list shows all containers you've created.
  • docker rm cups removes the cups container (if it's not running).
  • docker image list shows the software images that the containers are using.
  • docker rmi olbat/cupsd will remove the olbat/cupsd image, but only if the cups container that is based on it has been removed (and stopped).
  • docker exec cups ls /etc/cups will execute a command inside the container. You can execute /bin/sh or /bin/bash to explore inside the container machine.
  • ctop is a nice tool that will show you all containers and let you start/stop/restart them.

Preparing for using Docker

There are a couple of things you need to add to a fresh Debian install in order to use Docker:

  • docker, obviously, the package on Debian is called docker.io.
  • ctop is a nice CLI tool that shows your containers and lets you do stuff with them (stop, start, restart, enter shell, view logs etc.) It's basically a simple CLI version of Portainer (which I never bothered installing after all).

The following tools are indirectly related to services inside docker containers:

  • vainfo will verify that GPU-accelerated video encoding/decoding is working for AMD and Intel GPUs. This will be useful for many media streaming containers. See the Arch wiki for more.
  • avahi (which is avahi-daemon on Debian) and avahi-dnsconfd will help autodiscover some services on LAN between Linux machines. Only applicable if you have more than one Linux machine on your LAN, of course, and it's only relevant to some services (eg. CUPS).

Some tips about Docker

Should you use Docker or Podman? If you're a beginner just use Docker. It's a lot simpler. Yes, there are good reasons to use Podman but you don't need the extra headache when you're starting out. You will be able to transition into Podman easier later.

Use restart: "always" in your compose yaml's and save yourself unnecessary trouble. Some people try to micromanage their containers and end up writing sysctl scripts for each of them and so on and so forth. With this restart policy your containers will stay stopped if stopped manually, but will start each time the docker daemon [re]starts, which most likely means at boot, which is probably all you want right now.

The one docker issue that will give you the most trouble is mapping users from the real machine to the container machine and back. You want the service in the container to run as a certain user, and maybe you want to give it access to some files or devices on the real machine too. But some docker images were made by people who apparently don't understand how Linux permissions work. A good docker image will let you specify what users and groups it needs to work with (emby/embyserver is a very good example). A bad image will make up some UID and GID that's completely unrelated to anything on your machine and give you no way to change them; for such images you can try to force them to use root (UID and GID 0) but that negates some of the benefits a container was supposed to give you. So when looking for images see if they have a description, and if it mentions any UID and GID mapping. Otherwise you will probably have a bad time.

How do you find (good) docker images? On hub.docker.com, just search for what you need (eg. "cups"). I suggest sorting images by most recently updated, and have a look at how many downloads and stars it has, too. It's also very helpful if they have a description and instructions.

One last thing before we get to the good stuff. I made a dir on my RAID array, /mnt/array/docker where I make one subdir for each service (eg. /mnt/array/docker/cups), where I keep the magical yaml (compose.yaml) for each service, and sometimes I map config files from the container, so they persist even if the container is deleted. I also use git in those dirs to keep track of the changes to the yaml files.

Using Emby in a Docker container

Emby is a media server that you use to index movies and series and watch them remotely on TV or your phone and tablet. Some other popular alternatives are Plex, Jellyfin and Serviio. See this comparison chart.

Here's the docker-compose.yaml, explanations below:

version: "2.3"
services:
  emby:
    image: emby/embyserver
    container_name: emby
    #runtime: nvidia # for NVIDIA GPUs
    #network_mode: host # if you need DLNA or Wake-on-Lan
    environment:
      - UID=1000 # The UID to run emby as
      - GID=100 # The GID to run emby as
      - GIDLIST=100,44,105 # extra groups for /dev/dri/* devices
    volumes:
      - "./data:/config" # emby data dir (note the dot at the start)
      - "/mnt/nas/array/multimedia:/mnt/nas/array/multimedia"
    ports:
      - "8096:8096/tcp" # HTTP port
      - "8920:8920/tcp" # HTTPS port
    devices:
      - "/dev/dri:/dev/dri" # VAAPI/NVDEC/NVENC render nodes
    restart: always
  • version is the compose yaml specification version. Don't worry about this.
  • services and emby defines the service for this container.
  • image indicates what image to download from the docker hub.
  • container_name will name your container, normally you'd want this to match your service (and for me the dir I put this in).
  • runtime is only relevant if you have an Nvidia GPU, for accelerated transcoding. Mine is Intel so... More details on the emby image description.
  • network_mode: host will expose the container networking directly to the host machine. In this case you don't need to manually map the ports anymore. As it says, this is only needed for some special stuff like DLNA or WoL (and not even then, I achieve DLNA for example with BubbleUPnP Server without resorting to host mode).
  • environment does what I mentioned before. This is a very nicely behaved and well written docker image that not only lets you map the primary UID and GID but also adds a list of extra GUIDs because it knows we need to access /dev devices that are owned by 3rd party users like video and render. Very nice.
  • volumes maps dirs or files from the local real machine to the container. The config dir holds everything about Emby (settings, cache, data) so I map it outside of the container to keep it. When I installed this container I pointed it to the location of my old Emby stuff from the previous install and It Just Worked.
  • devices similarly maps device files.
  • ports maps the network ports that the app is listening on.
  • restart remember what I said about this above.

Using Deluge in a Docker container

Let's look at another nicely made Docker image. Deluge is a BitTorrent client, what we put in the Docker container is actually just the server part. The server can deal with the uploads/downloads but needs an UI app to manage it. The UI apps can be installed on your phone for example (I like Transdroid) but the Deluge server also includes a web interface on port 8112.

version: "2.1"
services:
  deluge:
    image: lscr.io/linuxserver/deluge:latest
    container_name: deluge
    environment:
      - PUID=1000
      - PGID=1000
      - DELUGE_LOGLEVEL=error
    volumes:
      - "./config:/config" # mind the dot at the start
      - "/mnt/nas/array/deluge:/downloads"
    ports:
      - "8112:8112/tcp" # web UI
      - "60000:60000/tcp" # BT transfers
      - "60000:60000/udp" # BT transfers
      - "58846:58846/tcp" # daemon remote control (for Transdroid)
    restart: always

Most of this is covered above with Emby so I won't repeat everything, just the important distinctions:

  • Notice how environment lets you choose what UID and GUID to work as.
  • I use volumes to map out the dir with the actual downloads, as well as map all the Deluge config dir locally so I can save it across container resets/reinstalls.
  • The ports need to be defined in the deluge config (which you can do via the web UI or edit the config directly) before you map them here. IIRC these are the defaults but please check.

Using Navidrome in a Docker container

Navidrome is a music indexer and streaming server (sort of like your own Spotify). It's follows the Subsonic spec so any client app that works with Subsonic will work with Navidrome (I like Substreamer). It also includes a web UI.

version: "3"
services:
  navidrome:
    image: deluan/navidrome:latest
    container_name: navidrome
    environment:
      ND_SCANSCHEDULE: 1h
      ND_LOGLEVEL: info
      ND_BASEURL: ""
      ND_PORT: 4533
      ND_DATAFOLDER: /data
      ND_MUSICFOLDER: /music
    volumes:
      - "./data:/data"
      - "/mnt/nas/array/music:/music:ro"
    ports:
      - "4533:4533/tcp"
    restart: "always"

Again, mostly self-explanatory:

  • Environment settings are nice but this image stopped short of allowing UID customization and just said fuck it and ran as root by default. Nothing to do here, other than go look for a nicer image.
  • I mapped the data dir locally so I preserve it between resets, and the music is shared read-only.

Using BubbleUPnPServer in a Docker container

This server can do some interesting things. Its bread and butter is DLNA. It has a companion Android app called, you guessed it, BubbleUPnP, which acts as a DLNA controller. The server part here can do local transcoding so the Android phone doesn't have to (subject to some caveats, for example the phone, the DLNA source and the Bubble server need to be on the same LAN; and it can only transcode one stream at a time).

It can also identify media providers (like Emby or Plex) on the LAN and media renderers (like Chromecast or Home Mini speaker) and DLNA-enables them so they appear in the Bubble app as well as on other DLNA-aware devices.

version: "3.3"
services:
  deluge:
    image: bubblesoftapps/bubbleupnpserver-openj9-leap
    container_name: bubbleupnpserver
    network_mode: "host"
    user: "0:0"
    devices:
      - "/dev/dri:/dev/dri:rw"
    volumes:
      - "./data/configuration.xml:/opt/bubbleupnpserver/configuration.xml:rw"
    restart: "always"
  • network_mode is "host" because this server needs to interact with lots of things on the LAN automagically.
  • user forces the server to run as root. The image uses a completely made up UID and GID, there's no way to customize it, and it needs to access /dev/dri which are restricted to video and render groups to access GPU-accelerated transcoding. So using root is the only solution here (short of looking for a nicer image).
  • I map the configuration file outside the container so it's saved across reset/reinstalls.

Using Samba in a Docker container

Normally I'd install samba on the host machine but Debian wanted me to install like 30 packages for it so I think that's a valid reason to use a container.

version: "2.3"
services:
  samba:
    image: twistify/anonymous-samba
    container_name: samba
    volumes:
      - "./etc/samba:/etc/samba:ro"
      - "/mnt/nas/array:/mnt/nas/array"
    ports:
      - "445:445/tcp" # SMB
      - "139:139/tcp" # NetBIOS
    restart: "always"

Normally this should be a simple enough setup, and it is simple as far as docker is concerned. Map some ports, map the config files, map the array so you can give out shares, done. But the image doesn't offer UID customization and just runs as root.

For reference I give the /etc/samba/smb.conf here too because I know it's tricky. This one only offers anonymous read-only shares, which mainly worked out of the box (hence why I stuck to this image in spite of the root thing).

[global]
   workgroup = WORKGROUP
   log file = /dev/stdout
   security = user
   map to guest = Bad User
   log level = 2
   browseable = yes
   read only = yes
   create mask = 666
   directory mask = 555
   guest ok = yes
   guest only = yes
   force user = root

[iso]
path=/mnt/nas/multimedia/iso

You can add your own shares aside from [iso], as long as the paths are mapped in the yaml.

Notice the ugly use of root in the Samba config too.

Using CUPS inside a Docker container

CUPS is a printer server, which I need because my printer is connected via USB to the server and I want to be able to print from my desktop machine.

version: "2.3"
services:
  cups:
    image: aguslr/cups:latest
    container_name: cups
    privileged: "yes"
    environment:
      - CUPS_USER=admin
      - CUPS_PASS=admin
    volumes:
      - "/dev/bus/usb:/dev/bus/usb" # keep this under volumes, not devices
      - "/run/dbus:/run/dbus"
    ports:
      - "631:631/tcp" # CUPS
    restart: "always"

The docker setup is not terribly complicated if you overlook things like /dev/bus/usb needing to be a volume mapping not a device mapping, or the privileged mode.

CUPS is complicated because it's a complex beast, so I'll try to add some pointers below (mostly unrelated to docker):

  • You can use lpstat -p inside the host to check if CUPS know about your printer, and /usr/lib/cups/backend/usb to check if it knows about the USB printer in particular.
  • You need CUPS on both the server and the desktop machine you want to print from. You need to add the printer on both of them. The CUPS interface will be at :631 on both of them, for printer management on the server the user+pass is admin:admin as you can see above, on the desktop machine God only knows (typically "root" and its password, or the password of the main user).
  • So the server CUPS will probably detect the USB printer and have drivers for it, this image did for mine (after I figured out the USB bus snafu). You need to mark the printer as shared!
  • ...but in order for the desktop machine to detect the printer you need to do one more thing: install Avahi daemon and dnsconfd packages on both machines because that's the stuff that actually makes it easy for the desktop machine to autodetect the remote printer.
  • ...and don't rely on the drivers from the server, the desktop machine needs its own drivers, which it may or may not have. For my printer (Brother HL-2030) I had to install an AUR package on desktop – and then the driver showed up when setting up the printer in the desktop CUPS.

See you next time with more docker recipes! As usual any and all comments and suggestions are welcome, including "omg you're so dumb, that thing could be done easier like this".

 

Upgrading a self-hosted server (2)

Adding a secondary disk to the machine

I'd like to have the old system around while tinkering with the new one, just in case something goes south. Also the old system is full of config files and scripts that are still useful.

I was planning to use a spare SSD I had lying around but it was a bit more involved than I'd expected. The machine has two M.2 slots with the old system disk occupying one of them, and 6 SATA ports on the motherboard, being used by the 6 HDDs. The spare SSD would need a 7th SATA port.

I could go get another M.2 but filling the second M.2 takes away one SATA channel, so I would be back to being one port short. 🤦

The solution was a PCI SATA expansion card which I happened to have around. Alternatively I could've disconnected one of the RAID arrays and free up some SATA ports that way.

Taking a system backup

So at this point I finally have two usable SSDs, one of which has the current system on it and is bootable.

What I'm proposing to do is make a 1-to-1 copy to the spare SSD and make it bootable too, then install Debian to the main SSD (the M.2).

It's simple because I used a single partition for the old system, so there are no multiple partitions for things like /home, /var, /tmp, swap etc.

  • Used fdisk to wipe the partition table on the backup SSD and create one primary Linux partition across the whole disk. In fdisk that basically means hitting Enter-Enter-Enter and accepting all defaults.
  • Used mkfs.ext4 -m5 to create an Ext4 filesystem on the backup SSD.
  • Mount the backup SSD partition and use cp -avx to copy the root filesystem to it.

I've seen many people recommend dd or clonezilla for the copy but that also duplicates the block ID, which you then need to change or there would be trouble. Others go for rsync or piping through tar but a simple cp -ax is perfectly fine for this.

Making the backup disk bootable

  • Used blkid to find the UUID of the backup SSD partition and change it in the backup's /etc/fstab.
  • If you have others partitions like /home, /var etc. you would do the same for each of them.
  • For swap I happen to use one that needs no special handling. It's a /swapfile in the root partition which I created with dd, formatted with mkswap, and mounted with /swapfile swap swap defaults 0 0 in /etc/fstab. That file was simply copied by cp so there's nothing else needed.

Next is installing grub on the backup SSD. You need to chroot into the SSD for this one as well as mount --bind /dev, /sys and /proc.

Here's an example for that last point, taken from this nice reddit post:

# mkdir /mnt/chroot
# mount /dev/SDD-partition-1 /mnt/chroot
# mount --bind /dev /mnt/chroot/dev
# mount --bind /proc /mnt/chroot/proc
# mount --bind /sys /mnt/chroot/sys
# mount --bind /tmp /mnt/chroot/tmp
# chroot /mnt/chroot
// Verify that `/etc/grub.d/30-os_prober` or similar is installed and executable
# update-grub
# grub-install /dev/SDD-whole-disk
# update-initramfs -u

I verified that I could actually boot into the backup disk before proceeding.

Installing Debian stable on the main disk

(I've decided to go with a regular install rather than debootstrap after all.)

Grabbed an amd64 ISO off Debian's website and put it on a flash stick. I used the graphical gnome-disk-utility app for that but you can also do dd bs=4M if=image.iso of=/dev/stick-device. Usual warnings apply – make sure it's the right disk, umount any pre-existing flash partition first etc.

Booting the flash stick should have been uneventful but the machine would not see it in BIOS or in the quick-boot BIOS menu so I had to poke around and disable some kind of secure feature (which will probably be different on your BIOS so good luck with that).

During the install I selected the usual things like keymap, timezone, the disk to install to (the M.2, not the SSD backup with the old system), chose to use the whole disk in one partition as before, created a user, disallowed login as root, and requested explicit installation of a SSH server. Networking works via DHCP so I didn't need to configure it.

First access

SSH'd into the machine using user + password, copied the public SSH key from my desktop machine into the Debian's ~/.ssh/authorized_keys, and then disabled password logins in /etc/ssh/sshd_config and service ssh restart. Made sure I could login with public key before disconnecting that root session.

Some other things I installed directly on the OS: ntpdate, joe, mc, apt-file, net-tools, htop, vainfo, curl, smartmontools, hdparm, rsync.

Mounting RAID arrays

The RAID arrays are MD (the Linux software driver) so they should have been already detected by the Linux kernel. A quick look at /proc/mdstat and /etc/mdadm/mdadm.conf confirms that the arrays have indeed been detected and configured and are running fine.

All that's left is to mount the arrays. After creating /mnt directories I added them to /etc/fstab with entries such as this:

UUID=array-uuid-here /mnt/nas/array ext4 rw,nosuid,nodev 0 0

...and then systemctl daemon-reload to pick up the new fstab right away, followed by a mount -a as root to mount the arrays.

Publishing the arrays over NFS

Last step in restoring basic functionality to my LAN is to publish the mounted arrays over NFS. My desktop machine can function fine without NFS but it's nice to have all the arrays available. I used nfs-kernel-server for this.

Please note that Debian and Ubuntu have a legacy issue with NFS where they fail to mount it after reboot. According to this discussion the most reliable solution is to make the service explicitly dependent on rpcbind by creating /etc/systemd/system/nfs-kernel-server.service.d/10-dep.conf with the following content:

[Unit]
Requires=rpcbind.service
After=rpcbind.service

To publish the arrays over NFS you add them to /etc/exports with entries such as this:

/mnt/nas/array desktop.lan(rw,async,secure,no_subtree_check,mp,no_root_squash)

Please note that the exports have to be done towards a client machine explicitly, either by name or IP. After a service nfs-kernel-server restart the NFS shares are ready for use.

Now on the client machine side you would most likely install nfs-utils and define the NFS shares in /etc/fstab like this:

nas:/mnt/nas/array /mnt/nas/array nfs vers=4,rw,hard,intr,noatime,timeo=10,retrans=2,retry=0,rsize=1048576,wsize=1048576 0 0

The first one (with nas: prefix) is the remote dir, the second is the local dir. I usually mirror the locations on the server and clients but you can of course use different ones.

Goals for the next episode

In the next episode I will attempt to install something non-essential, like Emby or Deluge, in a docker container. I intend to keep the system installed on the metal very basic, basically just ssh, nfs and docker over the barebones Debian install. Everything else should go into docker containers.

When working with docker I'd like to:

  • Keep the docker images on the system disk but all configurations on a RAID array.
  • Map all file access to a RAID array and network ports to the host. This includes things like databases and other persistent files, as long as they're important and not just cache or runtime data.

Basically the idea is that if I ever lose the system disk I should be able to simply reinstall a barebones Debian and redo the containers using the stored configs, and all the important mutable files would be on RAID.

See you next time, and meanwhile I appreciate any comments about what I could've done better as well as suggestions for next steps!

30
submitted 1 year ago* (last edited 1 year ago) by lemmyvore@feddit.nl to c/linux@lemmy.ml
 

I need a very simple method for non-advanced users to share each other's screen explicitly when they need help. They're running XFCE on Manjaro and the machines involved are using Tailscale. Edit: SSH access is also available, with key authentication.

I need something super simple because they are remote from me and from each other and any graphical setup will have to be assisted sight-unseen over phone. So ideally just (1) install something (which I can do for them over SSH), (2) pick something from the Applications menu and maybe (3) press a big "START" button.

It's also ok-ish if the remote capability is present all the time and I can connect without their explicit permission, but you can see why it would be best if they did something to enable it...

I've been looking for a solution but all I find is stuff that's way too complicated OR starts a new desktop session instead of showing the current one.

Edited: to clarify I'm not the one who will be remoting-in and to mention SSH is available.

TIA

 

I've been using Gandi for over 20 years, almost since it was founded. Since being acquired in 2019 by Montefiore Investment and this year by Total Webhosting Solutions their service have become more and more expensive and have finally priced me out.

For context, I administer a bunch of domains, mailboxes and HTML websites for my family and extended family, and I prefer services hosted in the EU because of GDPR and local availability.

This post is meant as a list of practical decisions in 2023 for the small time selfhoster. If anybody wants to comment on what Gandi (or rather TWS) is doing feel free to do so in the comments, I'm curious myself.

Prices I've mentioned use my country's VAT so will vary slightly for you.

Domain names

Domain names have always been a bit on the expensive side with Gandi but they used to include a lot of features for free with them (SSL, DNSSEC, mailboxes, a small static website, WHOIS privacy, local contact for TLDs that need it etc.) and what they added extra was proportional to the base TLD cost.

For the next renewal all my domains were slated to jump to €28 across the board. If you have domains with Gandi try adding some renewals to the cart and check in advance.

I had to look for an European registrar because I have lots of European ccTLDs that the usual suspects like Cloudflare and Porkbun don't support.

I'm moving to INWX.de and will be saving 25-60% per domain. This takes into account WHOIS privacy where needed for an extra 5€/domain (EU ccTLDs are private due to GDPR but we own a couple of TLDs too) as well as local contact services where required (price varies by country).

Email

I manage multiple mailboxes but they have low traffic and low storage requirements. Gandi will be offering them at €55/mailbox/year. I'm not questioning their pricing, 3-4€/month for email is common, but typically charged by email-focused services.

Anyway, this per-mailbox model would price us into hundreds of euros for resources that go 99% unused. I'm switching to Migadu.com, who allows unlimited domains and mailboxes (within common sense) under a single account and charges for the conflated storage space and emails sent/received across all mailboxes.

Migadu tiers start at 20€/year for 5GB and 200/20/day (soft limits).

Webhosting

We were using Gandi's smallest hosting package for about 100€/year, which was slated to jump to €135. Not an outlandish price for your typical PHP + MySQL hosting, especially since it had some VPS-like features. Then again the typical webhosting service would include a couple of mailboxes and some other goodies.

This was a good opportunity for us to reevaluate out hosting needs and realize we can ditch PHP+MySQL (if we really have to revisit it we'll consider VPS offers in the future). It's mostly static sites, image galleries and a bit of blogging. We've cached all our stuff as plain HTML/CSS/images and moved it to BunnyCDN.

Bunny lets you define a file bundle, gives you FTP access with a unique username+password, lets you pick the extent of replication, puts a CDN on top of it, and lets you point a domain name to it. Also throws a bunch of web server-ish features on top like rules/rewrites and Let's Encrypt SSL.

They actually offer more features than that but I've just mentioned the minimum you need for serving a bunch of static websites.

Bunny pricing starts at $0.01/GB (with a minimum of $1/month) and you pay as you go.

Nameservers

Since we're doing this I've taken the opportunity to dab into DNS. Turns out it's not that hard. There's only like half a dozen of commonly used DNS record types and everybody's helping you with them – email services like Migadu generate the email-related ones for you, registrars and managed DNS services generate the SOA for you, they have forms that tell you what fields are needed etc.

There are lots of managed DNS options. Registrars usually include nameservers and let you mess with the records so INWX was one choice. Bunny offers DNS service that integrates with their CDN. deSEC is a completely free service I'll be using as backup.

All of the above also offer APIs so a bash script will be taking care of dynamic DNS.

 

Upgrading a self-hosted server (1)

Welcome

Hi, I'm starting a series of posts that will follow the upgrades I'll be doing to a self-hosted machine that serves as NAS and also runs all kinds of self-hosted software. I'm lazy so it will probably take time, don't expect me to post too often.

About me: I've been using Linux exclusively for personal use (both desktop and servers) for about 20 years now. I've used several distributions over the years, I've built my own stuff from source (including kernels) and I've done Linux From Scratch. I'm not a Linux expert or professional sysadmin but I know my way around it, and I can learn what I don't know. So don't be afraid to make any suggestions no matter how complicated.

The current state of the machine

  • It's a PC using an i5 7400 CPU, has a built-in GPU with support for h264 hardware encoding and MPEG2, VP8, VP9 and HEVC hardware decoding (this will come in handy for video transcoding).
  • Only 4 GB of RAM, I have ordered a dual 2x16 GB kit.
  • The system drive is a Transcend M.2 SSD (32 GB). SATA rather than PCIe unfortunately but it will do fine for the time being.
  • The OS is Ubuntu Server 16.04 LTS using Expanded Security Maintenance for updates.
  • It's currently running SSH, NFS, Samba, CUPS, OpenVPN, Emby and Deluge on bare metal. Some of them come from distro packages, some from binary releases straight from the developer.
  • There are 6 HDDs forming 3 pairs of RAID 1 arrays. 6 drives was a limit I chose from the beginning, and the case and motherboard were chosen accordingly (cage for 6 drives and 6 SATA connectors).
  • My ISP provides a public dynamic IP and allows port forwards.
  • I have a router that I've recently upgraded to the latest OpenWRT so it also runs Linux, can install packages, it has a web admin interface etc. and can do some interesting stuff.

What I'd like to do

  • Increase the RAM to 32 GB.
  • Stick with a Linux distro, as opposed to a NAS-tailored OS, Unraid etc.
  • Install Debian Stable on a SSD, most likely via debootstrap from the Ubuntu system.
  • Add a GRUB menu entry that makes a passthrough to the other system, so I can keep them both around for a while.
  • Use docker-compose and possibly Portainer for as many of the services as it makes sense. Not sure if it's worth bothering to make containers for things like SSH, NFS, Samba.
  • Add more services. I'd like to try Jellyfin, NextCloud and other stuff (trying to degoogle for example).
  • I'd like to find a better solution for accessing services from outside the LAN. Currently using OpenVPN which is nice for individual devices but gets complicated when you want an entire remote LAN to be able to access (to allow smart TVs or Chromecast to use Emby/Jellyfin for example). I'm hoping Authelia + reverse proxy will be able to help with this.

What I'm not interested in

  • Not interested in using Plex. I've used it for a couple of years, it's a fine piece of software but I don't like the fact they now mandate access through their server or injecting ads.
  • Not interested in changing the filesystem or the RAID setup for the HDDs. RAID 1 pairs give me enough redundancy. The HDD upgrades are very simple. I'm fine with losing 50% of capacity.

Any and all suggestions and comments are welcome! Even if they're about things I said I'm not interested in. It's always possible there are things I haven't considered.

view more: next ›