this post was submitted on 25 Jan 2024
33 points (100.0% liked)

Linux

47290 readers
2121 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
AgreeableLandscape@lemmy.ml
nooter692@lemmy.ml
MarcellusDrum@lemmy.ml
cypherpunks@lemmy.ml
cyclohexane@lemmy.ml
d3Xt3r@lemmy.nz
33
Does Nix's break from FHS cause problems? (lemy.lol)
submitted 7 months ago by matcha_addict@lemy.lol to c/linux@lemmy.ml
30 comments fedilink hide all child comments
 

I know that a lot of what Nix does is working around its break from FHS, but I can imagine there are still things that seep through. Are there any unsolvable problems due to this?

I saw on this post that it is possible to use FHS on Nix. Does this solve all potential issues then?

all 32 comments
sorted by: hot top controversial new old
[–] chaorace@lemmy.sdf.org 13 points 7 months ago (1 children)

You may be interested in reading this post about the process of packaging Steam.

tl;dr: It's mostly an annoyance reserved for packagers to deal with. Dynamically linked executables can be patched in a fairly universal fashion to work without FHS, so that's the go-to approach. If the executable is statically linked, the package may have to ship a source patch instead. If the executable is statically linked & close-source, the packagers are forced to resort to simulating an FHS environment via chroot.

[–] matcha_addict@lemy.lol 1 points 7 months ago (2 children)

So that means packaging software for nix is a pain, compared to, say, gentoo or arch's AUR, but only for a small subset of packages.

I'll keep this in mind as I'm exploring if I should switch from Gentoo.

[–] hackeryarn@lemmy.world 6 points 7 months ago (1 children)

I would say it’s actually easier in many cases. Nix has really fantastic packaging tooling. You do have to learn a bit of the nix language, however (not become an expert).

The issue comes when trying to build from source. In most other distros, ou just follow the readme. In nix, you have to package it.

[–] matcha_addict@lemy.lol 1 points 7 months ago (2 children)

If I am packaging software for gentoo, all I have to do is translate the build instructions from the project's documentation to gentoo's package recipe. In nix, it seems that it is not that simple and you'll have to do some exploration. Am I wrong?

[–] pastermil@sh.itjust.works 3 points 7 months ago

It's just that most (if not all) build system in the source code package would assume some level of FHS compliance.

For example, they would install:

  • executables under /bin /usr/bin
  • libraries under /lib or /usr/lib
  • sysconfigs under /etc
  • manpages under /usr/share/man
  • and so on...

These build systems would include options to change these, but you'd then have to change all these values to adapt to nix structure. While it's all been done by the nix package maintainers, you'd have to do all that if you're to come up with a new package.

In the FHS compliant distros, the maintainers wouldn't need to change anything since these values are already set to the values they want (there are actually minor details they'd change, but that's another topic).

[–] Atemu@lemmy.ml 2 points 7 months ago (1 children)

If I am packaging software for gentoo, all I have to do is translate the build instructions from the project’s documentation to gentoo’s package recipe.

It's the same for Nixpkgs.

In nix, it seems that it is not that simple and you’ll have to do some exploration. Am I wrong?

In well behaved build systems, it's likely easier to package than most other distros. If it's not as well behaved you will have to do some "exploration" and the complexity can get quite out of control if the build system is exceptionally terrible.

Here is the package for the GNU hello program which uses a well-behaved build system:

https://github.com/NixOS/nixpkgs/blob/94b11073db6a7ca5733bc2d45378d800d9542975/pkgs/by-name/he/hello/package.nix

If you ignore the optional passthru.tests, this is very simple. You provide metadata, sources etc. to the generic mkDerivation function and that's it. The most complex non-standard thing this derivation does is enable the build system's tests.

You don't even need to run the provided build instructions because Nixpkgs' stdenv abstracts those away. If it finds a makefile, it'll automatically run make and make install with the correct flags for instance. Same for other standard build systems; if you pass cmake into nativeBuildInputs, it'll attempt to build, install, check etc. using cmake's standardised interfaces.

If the build system is poorly behaved however (like for instance Anki's), you will have to get into the weeds and do some rather advanced things:

https://github.com/NixOS/nixpkgs/blob/94b11073db6a7ca5733bc2d45378d800d9542975/pkgs/games/anki/default.nix

Luckily though, most packages aren't like this.

[–] matcha_addict@lemy.lol 2 points 7 months ago

Thank you for the thorough comment!

[–] sxan@midwest.social 6 points 7 months ago (1 children)

Gobolinux enters the room.

Gobo's been around and doing its alternative thing, successfully, for 20 years, so no. It's not a problem.

[–] mvirts@lemmy.world 1 points 7 months ago

Oh I remember trying this, I should give it another go!

[–] priapus@sh.itjust.works 4 points 7 months ago (1 children)

Most apps work fine, apps that don't get put in a FHS sandbox.

[–] Chewy7324@discuss.tchncs.de 2 points 7 months ago (1 children)

What do you use for a FHS sandbox?

distrobox and steam-run worked for me at times.

[–] priapus@sh.itjust.works 2 points 7 months ago

Those two and nix-alien.

[–] Lojcs@lemm.ee 3 points 7 months ago (4 children)

Why doesn't nix use fhs again?

[–] madmaurice@discuss.tchncs.de 14 points 7 months ago (1 children)

Nix installs derivations into separate folders. A derivation can be a package, but can also be other things like configuration files, scripts or sources for packages. Nix doesn't distinguish between these derivations by a name but rather by a hash created from their build instructions.

For example two instances of the same package with a different version are two different derivations and thus nix can have both package versions installed without them interfering with each other. But this goes beyond just a package version. It is e.g. possible to have the same package with the same version but different patches applied, or relying on different versions of dependencies. Since their build instructions differ both can be installed simultaneously.

This approach grants a variety of advantages. For example upgrading your NixOS system just installs new derivations of packages and configuration files that have changed, while keeping previous derivations until they're garbage collected at a later time. This allows you to switch freely between both iterations of your system, for example if an update causes issues you can just revert back to before an update easily. Another advantage is that an unprivileged user can install packages they need without interfering with the rest of the system, for example an older python version or a newer one, or some software they want but the system does not provide.

The price of having this kind of isolation between packages is that nixos cannot install binaries and libraries into common locations. Effectively /usr/bin only contains the env binary. If you're familiar with shell scripting you might have run into lines such as #!/usr/bin/env bash. This env util will essentially search bash in your PATH variable and start it. Lines like #!/bin/bash however will not work, because there's no bash installed in that location.

Another case where a missing fhs is a problem is when using pre-compiled binaries. In contrast to binaries built through nix, which have their required libraries hardcoded as absolute paths, pre-compiled binaries you download usually only contain the name of the library they need, which works in a conventional fhs environment, because these libraries tend to be found in /libor /usr/lib. On NixOS neither of those are present. There two solutions to this. Either you create an fhs environment by listing the set of derivations to be symlinked into a chroot environment which mimics an FHS. Or you can install https://github.com/Mic92/nix-ld which automatically finds the required libraries the nix way if you start such a binary. There's also steam-run which installs an fhs with most of the dependencies necessary to start Linux games from Steam.

[–] Lojcs@lemm.ee 1 points 7 months ago (2 children)

Either you create an fhs environment by listing the set of derivations to be symlinked into a chroot environment which mimics an FHS.

Why isn't this done on the actual system and by default? That would make it fhs compliant, no?

[–] hallettj@beehaw.org 7 points 7 months ago

If you put an FHS on the actual system you wouldn't be able to install multiple versions of the same package, updates wouldn't be atomic - you wouldn't get the big selling points of Nix.

[–] palebluethought@lemmy.world 6 points 7 months ago (1 children)

If your system uses 3 different Pythons as dependencies of different packages, which one gets to be /usr/bin/python?

[–] Lojcs@lemm.ee 2 points 7 months ago (2 children)

The most recent one by default unless another is manually chosen. The nix packages can keep using their specific versions directly

[–] palebluethought@lemmy.world 8 points 7 months ago* (last edited 7 months ago)

Now you'll have a zillion users trying to install software in ways that violate all the assumptions that NixOS operates on, but which are still tightly coupled to your NixOS config. Now updates to your system, or even seemingly unrelated config changes (through some transitive dependency chain) can easily break that software.

So now we've basically removed half the advantages that motivate Nix/OS in the first place, and when stuff breaks it will look like it's Nix's fault, even if it isn't.

On the other hand, nixpkgs is already the most comprehensive repository of system software out there, and for 99% of packages Nixifying it is pretty trivial. Hell, my NixOS config does that for 3 different GitHub repos right inline in my config.nix

[–] madmaurice@discuss.tchncs.de 1 points 7 months ago (1 children)

Choosing the most recent one might be impossible if you have multiple installations of the same package with same version but different features enabled during the configure step.

[–] Lojcs@lemm.ee 0 points 7 months ago (1 children)

Conflict resolution is not an impossible task. You just need to have a sensible algorithm. I get that they don't want to do it lest people abuse it instead of using nix but there isn't a technical challenge that can't be overcome.

[–] madmaurice@discuss.tchncs.de 0 points 7 months ago (1 children)

Conflict resolution was not my point. Rather the question which the "most recent" between two almost identical installations...

[–] Lojcs@lemm.ee 0 points 7 months ago (1 children)

That's what I meant. How the default is chosen is irrelevant and is not my point. (You can pick the earliest installed among the latest version for example) The point is, it can be done and isn't a technical challenge.

[–] madmaurice@discuss.tchncs.de 0 points 7 months ago* (last edited 7 months ago) (1 children)

It's not that it's hard to do. It's that it goes directly against the idea of NixOS since it breaks the separation. With NixOS I can start a shell in a different iteration of my system without switching over the whole system. If I had all my software installed into standard places, that shell might find things it's not supposed to find.

Bottom line is: Most things work on NixOS out of the box. The PATH variable is adjusted accordingly to what a program is supposed to find, which in my opinion is perfectly reasonable and enough for software to find other software. The dynamic library paths are hardcoded as absolute paths, so software can find it's libraries. There's a special dynamic loader for binaries that don't adhere to this. And if you really need an FHS compliant environment NixOS gives you the tools to create one in a sandbox.

You can either have the perks of NixOS or use an FHS compliant distro. That's your choice.

[–] Lojcs@lemm.ee 0 points 7 months ago (1 children)

It's not that it's hard to do. It's that it goes directly against the idea of NixOS since it breaks the separation.

That's what I said:

I get that they don't want to do it lest people abuse it instead of using nix but there isn't a technical challenge that can't be overcome.

[–] madmaurice@discuss.tchncs.de 2 points 7 months ago

Imho there's a difference between "people abuse it" and "it is possible for programs to use software that they shouldn't even find". Anyway I noticed just now you weren't the one to actually ask the initial question of whether it's technically possible, so I apologize for not noticing this earlier. However I think it's a meaningless endeavor to ponder whether or not it's possible when that fact is irrelevant.

[–] fl42v@lemmy.ml 3 points 7 months ago

So that it's possible for different versions of software/libs to coexist on the same system

[–] OmnipotentEntity@beehaw.org 1 points 7 months ago

Because the nix package manager places all system packages under /nix/store/uniquehash-packagename-version/

Where the unique hash is obtained via a Merkel tree of all the inputs. So in particular, binaries and libraries exist underneath those directories, not in the places you would expect from FHS.

In order to make the system actually work, environment variables are set up and executables are patched to refer to specific paths within the Nix Store.

[–] priapus@sh.itjust.works 1 points 7 months ago

A lot of features of the Nix package manager, such as having multiple versions of a package installed, don't work with FHS.

[–] mvirts@lemmy.world 1 points 7 months ago (1 children)

Nothing unsolvable, but it can be a pain when you want to run something not in nixpkgs. My solution is to have Ubuntu on a separate partition, and I was using docker to solve this problem for a while but have moved away from it.