[bug#45692] [PATCH 4/4] gnu: Add ZFS service.

[raid5atemyhomework]

This new version is dependent on https://issues.guix.info/45723 and 
https://issues.guix.info/45722

New features:

* We can now delay pool importation after particular mapped devices are opened 
or file systems are mounted.  This is useful if you want to make a ZFS pool out 
of LUKS containers, or mount an encrypted ZFS dataset with the encryption key 
in another (hopefully also encrypted) filesystem.
* We can now put other non-ZFS filesystems inside ZVOLs, block devices provided 
by a ZFS pool, and have them mounted after ZFS has scanned pools and opened 
such ZVOLs for use by the ystem.

It's now possible to have a bunch of LUKS containers in a ZPOOL that provides a 
ZVOL that contains an EXT4 filesystem (though why would you do that, just use 
builtin ZFS encryption).

This uses my [PATCH 1/4] instead of https://issues.guix.info/42193#4.  Should I 
use the latter instead?  What would get ZFS on Guix faster?

I decided to remove ZFS parameter options, because I don't know whether I 
should wait for https://issues.guix.info/42193 and how long that would take.


>From d0e095136cd471f083d92416f12ca22b47301f31 Mon Sep 17 00:00:00 2001
From: raid5atemyhomework <raid5atemyhomework@protonmail.com>
Date: Wed, 6 Jan 2021 09:24:20 +0800
Subject: [PATCH 4/4] gnu: Add service to install ZFS.

* gnu/services/file-systems.scm: New file.
(zfs-service-type): New variable.
(<zfs-configuration>): New type.
(%zfs-zvol-dependency): New variable.
* gnu/local.mk: Add gnu/services/file-systems.scm.
* gnu/services/base.scm (dependency->shepherd-service-name): Export.
* doc/guix.texi (ZFS file system): New subsection.
---
 doc/guix.texi                 | 107 +++++++++++++++++++++
 gnu/local.mk                  |   1 +
 gnu/services/base.scm         |   4 +-
 gnu/services/file-systems.scm | 174 ++++++++++++++++++++++++++++++++++
 4 files changed, 285 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/file-systems.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index a31d355780..714274b8c9 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -13886,6 +13886,113 @@ a file system declaration such as:
 compress-force=zstd,space_cache=v2"))
 @end lisp

+@node ZFS file system
+@subsection ZFS file system
+
+The ZFS file system has a license incompatible with the Linux kernel,
+and thus cannot be distributed with the kernel. However, as a user
+you have the right to do whatever you want on your own hardware,
+including download the ZFS source code, compile it, link it to your
+own private copy of Linux, and run it. You can even use the Guix build
+system to automate this.
+
+As a large and complex kernel module, ZFS on Linux has to be compiled
+with a specific version of the kernel. Often even the latest ZFS
+package available cannot be compiled with the latest Linux kernel
+package provided by Guix System. Thus, installing the @code{zfs}
+package is likely to fail.
+
+Instead, you have to use an older long-term-support Linux kernel.
+Do not use @code{linux-libre-lts} as the latest long-term-support
+kernel might be too new for the ZFS package; instead, explicitly
+select the version number, like @code{linux-libre-5.4}.
+
+Then, you have to modify your system configuration file and use the
+selected older kernel, and add the @code{zfs-service-type} service.
+
+@lisp
+(use-modules (gnu))
+(use-package-modules
+   ; @dots{}
+   linux)
+(use-service-modules
+   ; @dots{}
+   file-systems)
+
+(define my-kernel linux-libre-5.4)
+
+(operating-system
+  (kernel my-kernel)
+  ;; @dots{}
+  (services
+    (cons* (service zfs-service-type
+             (zfs-configuration
+               (kernel my-kernel)))
+           ; @dots{}
+           %desktop-services))
+  ;; @dots{}
+  )
+@end lisp
+
+@defvr {Scheme Variable} zfs-service-type
+This is the type of the service to compile and install OpenZFS to
+your operating system. It loads the ZFS module at startup, imports
+pools, mounts automounted ZFS datasets, installs the ZFS command
+line tools, and can provide module options for the ZFS module.  Its
+value must be a @code{zfs-configuration} record (see below).
+
+Here is an example use:
+
+@lisp
+(service zfs-service-type
+  (zfs-configuration
+    (kernel linux-libre-5.4)
+    (options '("zfs_arc_max=5000000000"))))
+@end lisp
+@end defvr
+
+@deftp {Data Type} zfs-configuration
+This data type represents the configuration of the ZFS service.
+The available fields are:
+
+@table @asis
+@item @code{kernel}
+The package of the Linux kernel to compile ZFS for. Required.
+
+@item @code{base-zfs} (default: @code{zfs})
+The ZFS package to use. It will be modified to use the indicated
+kernel.
+
+@item @code{options} (default: @code{'()})
+A list of string options to pass as options to the ZFS module.
+These will be put in a @file{/etc/modprobe.d/zfs.conf} file,
+for example setting this to @code{'("zfs_admin_snapshot=1"
+"zfs_trim_extent_bytes_min=0")} will create the following file:
+
+@example
+options zfs zfs_admin_snapshot=1 zfs_trim_extent_bytes_min=0
+@end example
+@end table
+@end deftp
+
+Once your system has been configured to include the ZFS service
+and you have restarted the system, you can manage ZFS pools and
+datasets with @code{zpool} and @code{zfs} commands.
+
+ZFS datasets with an appropriate @code{mountpoint} property will
+be automounted at startup after the root file system is started.
+Encrypted datasets that are automounted will cause boot to pause
+and prompt for the password to be provided on the console.
+
+It's possible to have a ZFS dataset as your @code{/home} by simply
+setting the @code{mountpoint} property. However, note that ZFS will
+refuse to mount over a non-empty directory, so if your root
+filesystem already has a non-empty @code{/home}, you should remove
+it and its contents, then restart the system.
+
+Having ZFS as a root filesystem or as @code{/boot} is not supported
+yet.
+
 @node Mapped Devices
 @section Mapped Devices

diff --git a/gnu/local.mk b/gnu/local.mk
index 1151d4642e..5aeb45c4c2 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -602,6 +602,7 @@ GNU_SYSTEM_MODULES =                                \
   %D%/services/dict.scm                                \
   %D%/services/dns.scm                         \
   %D%/services/docker.scm                      \
+  %D%/services/file-systems.scm                        \
   %D%/services/authentication.scm              \
   %D%/services/games.scm                       \
   %D%/services/ganeti.scm                      \
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index 13cfb6a8a2..ef3680583b 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -188,7 +188,9 @@

             references-file

-            %base-services))
+            %base-services
+
+            dependency->shepherd-service-name))

 ;;; Commentary:
 ;;;
diff --git a/gnu/services/file-systems.scm b/gnu/services/file-systems.scm
new file mode 100644
index 0000000000..9061ab9582
--- /dev/null
+++ b/gnu/services/file-systems.scm
@@ -0,0 +1,174 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 raid5atemyhomework <raid5atemyhomework@protonmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services file-systems)
+  #:use-module (gnu packages file-systems)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services linux)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system mapped-devices)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+  #:use-module (guix records)
+  #:export (zfs-service-type
+
+            zfs-configuration
+            zfs-configuration?
+            zfs-configuration-kernel
+            zfs-configuration-base-zfs
+            zfs-configuration-dependencies
+
+            %zfs-zvol-dependency))
+
+(define-record-type* <zfs-configuration>
+  zfs-configuration make-zfs-configuration zfs-configuration?
+  ; kernel you want to compile the base-zfs module for.
+  (kernel         zfs-configuration-kernel)
+  ; base package that will be compiled for the kernel
+  (base-zfs       zfs-configuration-base-zfs
+                  (default zfs))
+  ; list of <mapped-device> | <file-system> we should wait for,
+  ; before scanning for ZFS pools.
+  (dependencies   zfs-configuration-dependencies
+                  (default '())))
+
+;; This is a synthetic and unusable MAPPED-DEVICE; its only use
+;; is to be added as a (dependency ...) of some FILE-SYSTEM.
+(define %zfs-zvol-dependency
+  (mapped-device
+    (source '())
+    ;; The /* prevents naming conflict with non-ZFS device mappings,
+    ;; since it is not a valid name for mapped devices, and also
+    ;; implies "all zvols" in terms of globs.
+    (targets '("zvol/*"))
+    (type #f)))
+
+(define (make-zfs-package conf)
+  (let ((base-zfs (zfs-configuration-base-zfs conf))
+        (kernel   (zfs-configuration-kernel conf)))
+    (package
+      (inherit base-zfs)
+      (name (string-join (list (package-name base-zfs)
+                               "for"
+                               (package-name kernel)
+                               (package-version kernel)
+                               "version")
+                         "-"))
+      (arguments (cons* #:linux kernel (package-arguments base-zfs))))))
+
+(define (zfs-loadable-module conf)
+  (list (list (make-zfs-package conf) "module")))
+
+(define (zfs-shepherd-services conf)
+  (let* ((zfs-package     (make-zfs-package conf))
+         (zpool           (file-append zfs-package "/sbin/zpool"))
+         (zfs             (file-append zfs-package "/sbin/zfs"))
+         (zvol_wait       (file-append zfs-package "/bin/zvol_wait"))
+         (scheme-modules  `((srfi srfi-1)
+                            (srfi srfi-34)
+                            (srfi srfi-35)
+                            (rnrs io ports)
+                            ,@%default-modules)))
+
+    (define zfs-scan
+      (shepherd-service
+        (provision '(zfs-scan))
+        (documentation "Scans for ZFS pools.")
+        (requirement `(kernel-module-loader
+                       root-file-system
+                       ,@(map dependency->shepherd-service-name
+                              (zfs-configuration-dependencies conf))))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error importing zpools: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     ; TODO: optionally use a cachefile, for systems with 
dozens or
+                     ; hundreds of devices.
+                     (invoke/quiet #$zpool "import" "-a" "-N"))))
+        (stop #~(const #t))))
+
+    (define device-mapping-zvol/*
+      (shepherd-service
+        (provision '(device-mapping-zvol/*))
+        (documentation "Waits for ZFS ZVOL devices to appear.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error waiting for zvols: ~a~%"
+                                      (condition-message c))
+                              #f))
+                     (invoke/quiet #$zvol_wait))))
+        (stop #~(const #t))))
+
+    (define zfs-automount
+      (shepherd-service
+        (provision '(zfs-automount))
+        (documentation "Automounts ZFS datasets.")
+        (requirement '(zfs-scan))
+        (modules scheme-modules)
+        (start #~(lambda _
+                   (guard (c ((message-condition? c)
+                              (format (current-error-port)
+                                      "error automounting zfs: ~a~$")
+                              #f))
+                     ; (current-output-port) is typically connected to 
/dev/klog,
+                     ; so redirect it to (current-error-port) so that user can 
see
+                     ; prompts for passphrases on console
+                     (with-output-to-port (current-error-port)
+                       (lambda ()
+                         (invoke #$zfs "mount" "-a" "-l"))))))
+        (stop #~(lambda _
+                  ;; make sure we don't keep any ZFS mountpoints busy.
+                  (chdir "/")
+                  ;; unmount everything.
+                  (invoke/quiet #$zfs "unmount" "-a" "-f")))))
+
+    (list zfs-scan
+          device-mapping-zvol/*
+          zfs-automount)))
+
+(define zfs-service-type
+  (service-type (name 'zfs)
+                (extensions
+                  (list
+                    ; install the kernel module
+                    (service-extension kernel-loadable-module-service-type
+                                       zfs-loadable-module)
+                    ; load the kernel module
+                    (service-extension kernel-module-loader-service-type
+                                       (const '("zfs")))
+                    ; scan ZFS pools, automount filesystem, wait for zvols.
+                    (service-extension shepherd-root-service-type
+                                       zfs-shepherd-services)
+                    ; make sure automount occurs before file-systems target is 
reached
+                    (service-extension file-systems-target-service-type
+                                       (const '(zfs-automount)))
+                    ; install ZFS management tools
+                    (service-extension profile-service-type
+                                       (compose list make-zfs-package))
+                    ; install ZFS udev rules
+                    (service-extension udev-service-type
+                                       (compose list make-zfs-package))))
+                (description
+                  "Install ZFS, an advanced filesystem and volume manager.")))
--
2.30.0