[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
01/02: website: Add post about guix shell fhs option.
|
From: |
Ludovic Courtès |
|
Subject: |
01/02: website: Add post about guix shell fhs option. |
|
Date: |
Fri, 6 Jan 2023 06:59:26 -0500 (EST) |
civodul pushed a commit to branch master
in repository guix-artwork.
commit cd01aa11e320270de3e82007c56ec4d3eee4b6fc
Author: John Kehayias <john.kehayias@protonmail.com>
AuthorDate: Fri Dec 16 02:22:17 2022 -0500
website: Add post about guix shell fhs option.
* website/posts/guix-shell-fhs.md: New file.
website/static/blog/img/guix-shell-fhs.gif: New file.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
---
website/posts/guix-shell-fhs.md | 255 +++++++++++++++++++++++++++++
website/static/blog/img/guix-shell-fhs.gif | Bin 0 -> 306976 bytes
2 files changed, 255 insertions(+)
diff --git a/website/posts/guix-shell-fhs.md b/website/posts/guix-shell-fhs.md
new file mode 100644
index 0000000..dd2b13c
--- /dev/null
+++ b/website/posts/guix-shell-fhs.md
@@ -0,0 +1,255 @@
+title: The Filesystem Hierarchy Standard Comes to Guix Containers
+date: 2023-01-04 15:00
+author: John Kehayias
+tags: Software development, Containers
+---
+
+GNU Guix is different from most other GNU/Linux distributions and perhaps
nowhere is that
+more obvious than the organization of the filesystem: Guix does not conform to
the
+[Filesystem Hierarchy
Standard](https://refspecs.linuxfoundation.org/fhs.shtml) (FHS). In
+practical terms, this means there is no global `/lib` containing libraries,
`/bin`
+containing binaries,¹ and so on. This is very much at the core of how Guix
works and some
+of the convenient features, like per-user installation of programs (different
versions,
+for instance) and a declarative system configuration where the system is
determined from a
+configuration file.
+
+However, this also leads to a difference in how many pieces of software expect
their world
+to look like, relying on finding a library in `/lib` or an external tool in
`/bin`. When
+these are hard coded and not overcome with appropriate build options, we patch
code to
+refer to absolute paths in the store, like
+`/gnu/store/hrgqa7m498wfavq4awai3xz86ifkjxdr-grep-3.6/bin/grep`, to keep
everything
+consistently contained within the store.
+
+It all works great and is thanks to the hard work of everyone that has
contributed to
+Guix. But what if we need a more FHS-like environment for developing, testing,
or running
+a piece of software?
+
+To that end, we've [recently
+added](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=c7ba5f38b80433b040d3946b8fc0b1e8621ba30a)
(available in [Guix
1.4.0](https://guix.gnu.org/en/blog/2022/gnu-guix-1.4.0-released/))
+a new option for [`guix
+shell`](https://guix.gnu.org/en/manual/devel/en/html_node/Invoking-guix-shell.html)
+(previously called [`guix
+environment`](https://guix.gnu.org/en/blog/2021/from-guix-environment-to-guix-shell/)):
+`--emulate-fhs` (or `-F`). This option is used in conjunction with the
+[`--container`](https://guix.gnu.org/en/cookbook/en/html_node/Containers.html)
(or `-C`)
+option which creates an isolated, you guessed it, container. The new
`--emulate-fhs`
+option will set up an environment in the container that follows FHS
expectations, so that
+libraries are visible in `/lib` in the container, as an example.
+
+Here is a very simple example:
+```sh
+$ guix shell --container --emulate-fhs coreutils -- ls /bin | head
+[
+b2sum
+base32
+base64
+basename
+basenc
+cat
+catchsegv
+chcon
+chgrp
+```
+and
+```sh
+$ guix shell --container --emulate-fhs coreutils -- ls /lib | head
+Mcrt1.o
+Scrt1.o
+audit
+crt1.o
+crti.o
+crtn.o
+gconv
+gcrt1.o
+ld-2.33.so
+ld-linux-x86-64.so.2
+```
+
+Contrast that with `/bin` on a Guix system:
+```sh
+$ ls /bin -l
+
+total 4
+lrwxrwxrwx 1 root root 61 Dec 12 09:57 sh -> \
+ /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8/bin/sh*
+```
+and `/lib`
+```sh
+$ ls /lib
+ls: cannot access '/lib': No such file or directory
+```
+
+Or, if you like to see it more in motion, here's a gif (courtesy of Ludovic
Courtès):
+
+
+Additionally, for the more technically-minded, the [`glibc` used in this
+container](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=3d1d29e440910a99531b738f8f090de2cd4df9da)
+will read from a global cache in `/etc/ld.so.cache` contrary to the [behavior
in
+Guix](https://guix.gnu.org/blog/2021/taming-the-stat-storm-with-a-loader-cache/)
+otherwise. This can help ensure that libraries are found when querying the ld
cache or
+using the output of `ldconfig -p`, for example.
+
+There are several uses that spring to mind for such a container in Guix. For
developers,
+or those aspiring to hack on a project, this is a helpful tool when needing to
emulate a
+different (non-Guix) environment. For example, one could use this to more
easily follow
+build instructions meant for a general distribution, say when a Guix package
is not (yet)
+available or easy to write immediately.
+
+Another usage is to be able to use tools that don't really fit into Guix's
model, like
+ones that use pre-built binaries. There are many reasons why this is not ideal
and Guix
+strives to replace or supplement such tools, but practically speaking they can
be hard to
+avoid entirely. The FHS container helps bridge this gap, providing an isolated
and
+reproducible environment as needed.
+
+Users not interested in development will also find the FHS container useful.
For example,
+there may be software that is free and conforms to the [Free System
Distribution
+Guidelines](https://www.gnu.org/distros/free-system-distribution-guidelines)
(FSDG) Guix
+follows, yet is not feasible to be
+[packaged](https://hpc.guix.info/blog/2021/09/whats-in-a-package/) by our
standards.
+JavaScript and particularly [Electron](https://www.electronjs.org/)
applications are not
+yet packaged for Guix due to the
+[difficulties](https://dustycloud.org/blog/javascript-packaging-dystopia/) of
a properly
+source-based and bootstrapable approach in this ecosystem.
+
+As a more interesting example for this last point, let's dive right into a big
one: the
+popular [VSCodium](https://github.com/VSCodium/vscodium) editor. This is a
[freely
+licensed](https://github.com/VSCodium/vscodium#why-does-this-exist) build of
Microsoft's
+VS Code editor. This is based on Electron and pre-built
[AppImages](https://appimage.org/)
+are available. [Downloading](https://github.com/VSCodium/vscodium/releases)
and making the
+AppImage executable (with a `chmod +x`), we can run it in a container with
+```sh
+guix shell --container --network --emulate-fhs \
+ --development ungoogled-chromium gcc:lib \
+ --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --expose=$XAUTHORITY \
+ --preserve='^DBUS_' --expose=/var/run/dbus \
+ --expose=/sys/dev --expose=/sys/devices --expose=/dev/dri \
+ -- ./VSCodium-1.74.0.22342.glibc2.17-x86_64.AppImage
--appimage-extract-and-run
+```
+
+The second line is a handy cheat to get lots of libraries often needed for
graphical
+applications (development inputs of the package `ungoogled-chromium`) though
it can be
+overkill if the AppImage does actually bundle everything (they don't!). The
next line is
+for display on the host's X server, the one after for DBus communication, and
lastly
+exposing some of the host hardware for rendering. This last part may be
different on
+different hardware. That should do it, at least to see basic functionality of
VSCodium.
+Note that we can't run an AppImage without the `--appimage-extract-and-run`
option as it
+will want to use
[FUSE](https://www.kernel.org/doc/html/latest/filesystems/fuse.html) to
+mount the image which is not possible from the container.²
+
+The FHS container is also useful to be able to run the exact same binary as
anyone else,
+as you might want to for privacy reasons with the [Tor
+Browser](https://www.torproject.org/). While there is a long-standing [set of
+patches](https://issues.guix.gnu.org/42380) to build the Tor Browser from
source, with a
+container we can run the official build directly. After
+[downloading](https://www.torproject.org/download/), checking the
+[signature](https://support.torproject.org/tbb/how-to-verify-signature/), and
+[unpacking](https://tb-manual.torproject.org/installation/), we can launch the
Tor Browser
+from the root of the unpacked directory with:
+
+```sh
+guix shell --container --network --emulate-fhs \
+ --preserve='^DISPLAY$' --preserve='^XAUTHORITY$' --expose=$XAUTHORITY \
+ alsa-lib bash coreutils dbus-glib file gcc:lib \
+ grep gtk+ libcxx pciutils sed \
+ -- ./start-tor-browser.desktop -v
+```
+Here we've used a more minimal set of package inputs, rather than the
`ungoogled-chromium`
+trick above. Usually this is found through some trial and error, looking at
log output,
+maybe tracing, and sometimes from documentation. Though documentation of
needed packages
+often has some assumptions on what is already available on typical systems.
(Thanks to Jim
+Newsome for pointing out this example on the [guix-devel mailing
+list](https://lists.gnu.org/r/guix-devel/2022-12/msg00208.html).)
+
+Another example is to get the latest nightly builds of
[Rust](https://www.rust-lang.org/), via [`rustup`](https://rustup.rs/).
+```sh
+$ mkdir ~/temphome
+
+$ guix shell --network --container --emulate-fhs \
+ bash coreutils curl grep nss-certs gcc:lib gcc-toolchain \
+ pkg-config glib cairo atk pango@1.48.10 gdk-pixbuf gtk+ git \
+ --share=$HOME/temphome=$HOME
+
+~/temphome [env]$ curl --proto '=https' --tlsv1.2 -sSf <https://sh.rustup.rs>
| sh
+```
+
+First we created a `~/temphome` directory to use as `$HOME` in the container
and then
+included a bunch of libraries in the container for the next example.
+
+This will proceed without problem and we'll see
+```sh
+info: downloading installer
+
+Welcome to Rust!
+
+This will download and install the official compiler for the Rust
+programming language, and its package manager, Cargo.
+
+...
+
+Rust is installed now. Great!
+
+To get started you may need to restart your current shell.
+This would reload your PATH environment variable to include
+Cargo's bin directory ($HOME/.cargo/bin).
+
+To configure your current shell, run:
+source "$HOME/.cargo/env"
+```
+
+After updating the shells environment as instructed, we can see it all worked
+```sh
+~/temphome [env]$ rustc --version
+rustc 1.65.0 (897e37553 2022-11-02)
+```
+as Guix's current Rust is at 1.61.0 and we didn't even include Rust in the
+container, of course.
+
+Finally, we can build a Rust project of desktop widgets, [ElKowars wacky
widgets
+(eww)](https://github.com/elkowar/eww), following [their
+directions](https://elkowar.github.io/eww/). Ultimately this uses just the
standard `cargo
+build --release` and builds after downloading all the needed libraries.
+```sh
+~/temphome/eww [env]$ git clone https://github.com/elkowar/eww
+...
+~/temphome/eww [env]$ cd eww
+
+~/temphome/eww [env]$ cargo build --release
+info: syncing channel updates for 'nightly-2022-08-27-x86_64-unknown-linux-gnu'
+info: latest update on 2022-08-27, rust version 1.65.0-nightly (c07a8b4e0
2022-08-26)
+...
+
+Finished release [optimized] target(s) in 2m 06s
+```
+
+With this being a fresh container, you will need to make some directories that
normally
+exist, like `~/.config` and `~/.cache` in this case. For basic display
support, it is
+enough to add `--preserve='^DISPLAY$' --preserve='^XAUTHORITY$'
--expose=$XAUTHORITY` to
+the container launch options and run the first example widget in the
+[documentation](https://elkowar.github.io/eww/configuration.html).
+
+As we can see, with containers more generally we have to provide the right
inputs and
+options as the environment is completely specified at creation. Once you want
to run
+something that needs hardware from the host or to access host files, the
container becomes
+increasingly porous for more functionality. This is certainly a trade-off, but
one which
+we have agency with a container we wouldn't get otherwise.
+
+The FHS option provides another option to make a container in Guix to produce
other
+environments, even those with a vastly different philosophy of the root
filesystem! This
+is one more tool in the Guix toolbox for controlled and reproducible
environments that
+also let's us do some things we couldn't (easily) do otherwise.
+
+##### Notes
+
+¹ Other than a symlink for `sh` from the `bash`
+package, for compatibility reasons.
+
+² Actually, one can use `flatpak-spawn` from
+[`flatpak-xdg-utils`](https://github.com/flatpak/flatpak-xdg-utils/) to launch
something
+on the host and get the AppImage to mount itself. However, it is not visible
from the same
+container. Or, we can use a normal [mounting
+process](https://docs.appimage.org/user-guide/run-appimages.html#mount-an-appimage)
+outside of the container to inspect the contents, but AppImages will have an
offset. We
+can use the FHS container option to get this offset and then mount in one line
with `mount
+VSCodium-1.74.0.22342.glibc2.17-x86_64.AppImage <mountpoint> -o offset=$(guix
shell
+--container --emulate-fhs zlib --
./VSCodium-1.74.0.22342.glibc2.17-x86_64.AppImage
+--appimage-offset)`
diff --git a/website/static/blog/img/guix-shell-fhs.gif
b/website/static/blog/img/guix-shell-fhs.gif
new file mode 100644
index 0000000..352caec
Binary files /dev/null and b/website/static/blog/img/guix-shell-fhs.gif differ