Re: Stratification of GNU Guix into Independent Channels

[Denis 'GNUtoo' Carikli]

On Sat, 24 Dec 2022 03:49:30 +0000
"jgart" <jgart@dismail.de> wrote:

> Hi Guixers,
Hi,

> Should GNU Guix be a small core of packages (and services?)?
I think it would also make sense to consider first find the features we
want, and then look at how to get them, to see if that works better for
us.

For instance if the goal is to get faster updates, there might be
other things that could be done to improve the speed way more than
reducing the number of packages in the repository. 

For instance making sure that there are substitutes might improve the
speed a lot more. It might also be possible to optimize Guile more for
some tasks. For instance:
- Copying files on filesystems that have deduplication could be
  improved.
- Maybe it is possible to substitute more things (like the things it
  builds when we do guix pull) or at least rebuilding only some of them
  (the ones that changed between the last git revision and the new
  one). 
- It might also be possible to combine information from Guix weather
  with the current system or packages to see on which revision the
  packages ca be upgraded without having to recompile anything (unless
  we use non-substitutable packages) and update to that revision
  automatically.
- etc.

For some of these ideas, we would first need to benchmark somehow what
is taking time during updates though, to make sure that the
optimization really yields some gains.

In my case I've already a clear use case (basically a non-substitutable
package that consist of a huge data file) where optimizing Guile will
yield a lot of performance improvements.

If the idea is (also) to avoid using certain packages, it might make
more sense to have a system to do that inside the current Guix source
code. 

For instance a user might want to make sure not to ship compiled zfs
compiled kernel modules, though right now it's not a big issue as you
need to manually select either zfs-auto-snapshot or zfs to get it, no
other packages depend on it. So users usually won't accidentally install
it.

Another idea would be to have some flag to have only free culture
licensed packages (like to avoid cc-nd non-functional data).

Though I think that if we do something like that, it would complicate
things a lot. Since Guix is meant to be reproducible, we would need to
find a way to handle that kind of configuration in a reproducible way.

So far the configurability has been integrated in ways that are not
hidden for users. For instance if you use a package transformation, you
are aware of it. Packages specific to CPU Microarchitectures[1][2] also
need to be used specifically, and if I understand users often use
package transformation for that.

An easier way here would probably be to teach tools like guix lint to
enable to check certain things, for instance with 'guix lint
--extra-check=./my-checks.scm', or just to write a new tool or external
tools for that. We can already use the Guix repl in guile scripts.

Though here it would still be up to users to run the checks and try to
fix things (for instance by using package transformations), but at
least it would not complicate things in Guix.

As for weather or not switching to a layered approach would improve
development, some projects like openembedded/yocto did that. I've no
idea why exactly they did it and what turned out to work well or not for
their use cases, but to me the result looks really complex[3],
especially if we want to do something like that with Guix.

For instance we'd need to handle this layered approach in all the
infrastructure, tools, and so on. Users would also need to know where
to send patches, and if we try to autodetect that in the tools that
receive the patches, it might becomes error prone. And we'd also need
to make sure that layers stay compatible with each other. If everything
is split, that's harder to do.

I think that one of the big strength of Guix is precisely the fact that
everything is well integrated together. And as far as I understand,
here having a single repository makes it much easier to do that.

As I understand Guix maintainers do care about giving some time
to third party packages or channels to adjust for changes[4], though
with a single repository we can more easily do changes and test them
than if the changes would be scattered around multiple repositories.

A single repository also ensures that everything stay consistent
without much effort.

References:
-----------
[1]https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/maths.scm?id=fddf1dc3aba3176b6efc9e0be0918245665a6ebf#n2762
[2]https://hpc.guix.info/blog/2018/01/pre-built-binaries-vs-performance/
[3]https://layers.openembedded.org/layerindex/branch/master/layers/
[4]"We recommend waiting until the next Guix release is out though,
   which could be a month from now, so that your channel remains usable
   by those who pinned an older revision of the guix channel."[5]
[5]https://guix.gnu.org/en/blog/2021/the-big-change/

Denis.

pgphLZv_piCp1.pgp
Description: OpenPGP digital signature