[bug#45692] kernel-module-configuration-service for configuring kernel parameters

[raid5atemyhomework]

Hi Danny,

> > See also https://issues.guix.info/issue/42193 for an earlier attempt (which
> > is already very far--but it has a bug somewhere). There's also already a
> > kernel profile thing like you wrote in that patchset.
> > (Note that I would prefer there not to be a "LOAD?" in there because it
> > confuses loading the module (which is usually NOT started by user space
> > but by the kernel on its own) and confguring the module (which has to be
> > done by user space because it's specifying policy, not mechanism))
>
> Looks like that patchset was merged in, so basically I can just depend on 
> that? So the first patch in this patchset would be dropped?

No, sorry, my mistake, only one patch completely unrelated to the actual new 
service type was merged in.

In any case --- is this objection something that would block this patchset from 
being added to Guix? I appreciate that we should "do it right" --- but there's 
also an argument for "keep it simple", and the first patch in this patchset 
gets us a good part of the way to what is needed.

It seems that my patch is equivalent to the existing WIP 2/6 
`kernel-profile-service-type`, so maybe I can just steal that patch for now, 
write tests specific for it, and get on with running ZFS in production?

Can we at least get patch 3/4 of *this* patchset merged since it's a trivial 
bugfix?  How about patch 2/4, which is not so trivial, but does give some 
flexibility in case other filesystems want to be as ambitious as ZFS is?

I imagine a later, much more comprehensive 
`kernel-module-configuration-service-type` can be built on top of 
`kernel-profile-service-type`, `kernel-module-loader-service-type`, and a later 
`kernel-module-options-service-type`.  Then `zfs-service-type` can be modified 
to use that `kernel-module-configuration-service-type` instead of rolling its 
own bits and pieces.

----

On the other hand, if we want to think of `/` on ZFS, then we need a notion of 
kernel modules that are added to the `initrd` file, rather than to the `kernel` 
directory.

Diving into `gnu/system/linux-initrd.scm`, I note that we can provide a kernel 
package and a list of modules to copy from the kernel package to the `initrd` 
by a `(flat-linux-module-directory linux modules)` function.

I imagine that it would be possible to modify this as follows:

* Have `flat-linux-module-directory` accept a list of packages from which to 
find module names, not just a single package.
* Remove more code from `(operating-system-directory-base-entries os)`, and put 
the creation of the `"initrd"` file into a service, in much the same way that 
my patch does (unless there's an existing more-guixy way of putting files into 
`initrd`?).
  * Create a service type that gathers packages whose modules are to be added 
to the `initrd`, and if that list is non-empty, pass it to a new key 
`#:extra-linux-module-packages` of the `initrd` function.
    * `raw-initrd` would then pass that field together with the `linux` 
argument to the modified `flat-linux-module-directory` procedure.
  * Create another service type that gathers module names to be loaded at 
`boot` and appends them to the `#:linux-modules` to the `initrd` function.
  * Modify the `kernel-module-options-service-type` to pass in options via 
kernel command line always, maybe.

Then ZFS module can get loaded early, at boot, before the switch from `initrd` 
root to the "real" root.

* Create a service type that gathers additional pre-mount actions, which 
`raw-initrd` will accept as a new key `#:additional-pre-mount` and append to 
the `pre-mount` it passes to `boot-system`.
  * The `zfs-service-type` would then extend this service type to pass in an 
action to perform a `zpool import -a -N`, which makes ZFS scan for devices 
containing ZFS pools.
* Somehow figure out a static build for the ZFS package, so we can use a 
statically-linked `zpool` in the above.
  * The `zfs-service-type` already inherits a ZFS package from the given 
`base-package`, I imagine it would be possible to make further inheritance 
which modifies the build to be static.  It does require an additional build, 
though.  Maybe an additional `root-on-zfs?` field in `zfs-configuration` can 
gate this, so that we don't add the static build and extend the boot script if 
`(not root-on-zfs?)` (i.e. use `kernel-module-configuration-service-type` if 
`(not root-on-zfs?)`, else use the new put-it-in-the-initrd service types).
* Modify the various checks elsewhere so that ZFS poolnames can be passed as 
the `device` field of `file-system` records.  See 
https://issues.guix.gnu.org/45643#3
  * Then the installer can be modified so that if ZFS is included with the 
installer's profile, it can look at ZFS pools and offer to install to a ZFS 
pool instead of a `/dev` partition directly, then add `(file-system (device 
"rootpool") (mountpoint "/") (type "zfs"))` and the appropriate 
`zfs-service-type` and etc. and now we get `/` on ZFS.

But that can come later, much much later, I just need ZFS, *any* ZFS, before 
RAID5 eats more of my homework.

Thanks
raid5atemyhomework


> > But I guess the ZFS Linux kernel module can't be built-in into the kernel
> > anyway.
> > But that's a special case--in general, it's very much possible to make 
> > modules
> > built-in.
>
> ZFScan be built-in to the kernel, Ubuntu does it. You can't distribute it 
> like that (Ubuntu distributes it like that but presumably they have enough 
> lawyers to muddy the waters so that they can get away with it), but as the 
> documentation in this patch notes: the user has every right to do whatever 
> they want on the machine they own, including build a Linux kernel that has 
> ZFS built-in and run it, they just can't make that version available to 
> somebody else.
>
> So to go whole-hog, we would have a service that replaces the kernel package 
> and inserts kernel module sources in-tree somehow, then compiles Linux-libre 
> on the user's machine. That would probably be a lot more painful to install 
> ZFS with (the user has to recompile the whole kernel at each update of either 
> kernel or ZFS, whereas with a kernel module the user has to recompile just 
> the kernel module), so maybe kernel module is still better overall.
>
> > > -                         ;; You'd think we could've used 
> > > kernel-module-loader-service-type,
> > >
> > >
> >
> > Definitely.
> >
> > > -                         ;; but the kernel-module-loader shepherd 
> > > service is dependent on
> > >
> > >
> > > -                         ;; file-systems,
> > >
> > >
> >
> > Yes--but why is that dependent on 'file-systems ? Is it because it needs 
> > /proc ?
> > Or is it an oversight ? I would prefer to get rid of this dependency and 
> > then
> > use kernel-module-loader-service-type.
>
> Dunno --- one VM I tested, I removed the`zfs-scan-automount` shepherd service 
> from the `file-systems` target, and the VM still wouldn't boot, claiming a 
> stack overflow (the same error which I got when I was still trying to use 
> kernel-module-loader-service-type here). Or maybe I just got confused with 
> which VM was which, testing VMs wasn't a stress-free vacation. I just want 
> ZFS, because MD RAID5 ate my homework, this is getting tiresome...
>
> One thing I notice about `kernel-module-loader-service-type` is that it's not 
> instantiated in essential services, or indeed anywhere in Guix. A few 
> services do extend it. But my very rough understanding is that if you're 
> going to extend a service, it had better be instantiated once in the list of 
> services.
>
> In particular I note that the documentation for 
> `kernel-module-loader-service-type` shows an example where it uses `service` 
> to program the `kernel-module-loader-service-type`, not `simple-service`. 
> This suggests to me that `kernel-module-loader-service-type` is broken 
> because it's not in the list of essential services but is extensible. Maybe. 
> It's designed as an extensible service, but isn't instantiated at default. 
> Maybe that's what really bit me and not the shepherd circular dependency 
> loop? shrug
>
> > Also, this manual loading of kernel modules is not supposed to be the way to
> > do things in Linux. That a kernel module was compiled as a module is
> > an implementation detail--so Linux should (and usually does) automatically
> > load kernel modules the first time a device for them is accessed (after all,
> > how would user space know whether something is compiled as a module or
> > built-in--that would be too much to ask).
>
> So how do I get ZFS loaded? Note that the devices it targets are block 
> devices and it needs to scan for block devices that are formatted for ZFS. Do 
> other filesystems have some autoload rule?
>
> Thanks
> raid5atemyhomework