[bug#27855] [PATCH] gnu: Add rsync service.

[Oleg Pykhalov]

Hello Christopher,

Christopher Baines <address@hidden> writes:

>> >> > Yep, I think I just stopped writing the test after finding the
>> >> > issue with the PID file.
>> >> >
>> >> > I haven't looked in to how to fix this in the test, so if you
>> >> > could, that would be great. Otherwise, I'll probably have time
>> >> > to look at this again within a week or so.
>> >> >
>> >> > You'll probably need to refactor the test a bit, as at the
>> >> > moment, the information regarding the port isn't available where
>> >> > you run the commands.    
>> >> 
>> >> Of course I'll try.  By the way, how to run a “vm”?  Previous
>> >> method “./pre-inst-env guix system vm gnu/tests/rsync.scm” doesn't
>> >> work for me.  
>> >
>> > I'm guessing that you'll need to make the file evaluate (I'm not
>> > sure if that is the right word here) to an operating-system, e.g.
>> > put %rsync-os-with-port-2000 right at the bottom of the file, and
>> > then guix system vm should give you a start script that will start
>> > a VM for that OS.  
>> 
>> I did some work on rsync service:
>> 
>> - Fixed PID and synchronization to specific port.
>> - Merged two rsync oses in one with optional port.
>> - Added ports to rsync synchronization tests and change protocol from
>>   ssh to rsync.
>> - Added some logic to config: chroot (can use only root), user and
>>   group.
>> 
>> All tests passed successfully for me.
>
> Great :) Now that the tests pass at least, I don't see any reason to
> not merge this soonish.
>
> I've still done some thingking about how the configuration works
> though, and I've been considering a few ways of tweaking this so that
> its harder to break, and clear in how it works.
>
> One way I've managed to break the service so far is setting the user
> and group to root in the configuration. This causes the tests to fail,
> and in a odd way, as I think the problem is that the creation of
> the /var/run/rsync directory relies on the account service, but I'm
> guessing that the account service does nothing in this case, as root is
> already an account.
>
> One way of making this harder to break would be to explicitly create
> the necessary directories when this service is activated, e.g.:
>
>   (mkdir-p (dirname #$(rsync-configuration-pid-file config)))
>   (mkdir-p (dirname #$(rsync-configuration-lock-file config)))
>
> I think there is also the opportunity to make the service configuration
> clearer here, as considering the default port test, the default
> configuration says it will run as the rsync user, but the service will
> actually run as the root user.
>
> This could be improved by making the configuration more uncertain by
> default, e.g. user defaults to #f, which means the correct user is
> decided based on the port.
>
> Also on the subject of clarity, the use-chroot? option is something
> that can be specified, but the value might not be used. My preference
> would be to change the "logic" in the configuration file generation, to
> validation, e.g.:
>
>   (if (and use-chroot? (not (eq? user "root")))
>       (error "rsync-service: to run rsync in a chroot, the user must be 
> root"))
>
> The use-chroot? option might also benefit from making the user default
> to #f, as then the service could decide the user based on the port and
> use-chroot? settings, without contradicting the configuration.

I tried to apply all your suggestion, but I really don't know why
'rsync-configuration-user' is ignored (it's always set to "rsyncd").

I use 'pk' to debug this, but still confused.

--8<---------------cut here---------------start------------->8---
;;; (rsync-config-file-config #<<rsync-configuration> package: #<package 
address@hidden gnu/packages/rsync.scm:32 216af00> port-number: 581 pid-file: 
"/var/run/rsyncd/rsyncd.pid" lock-file: "/var/run/rsyncd/rsyncd.lock" log-file: 
"/var/log/rsyncd.log" use-chroot?: #f share-path: "/srv/rsyncd" share-comment: 
"Rsync share" read-only?: #f timeout: 300 user: "rsyncd" group: "rsyncd" uid: 
#f gid: #f>)
--8<---------------cut here---------------end--------------->8---

>From 4013506b89ecfd9cc5c59f3311dfa9b07bc140d3 Mon Sep 17 00:00:00 2001
From: Oleg Pykhalov <address@hidden>
Date: Thu, 27 Jul 2017 04:01:01 +0300
Subject: [PATCH] gnu: Add rsync service.

* doc/guix.texi (Incremental file transfer): Add documentation.
* gnu/services/rsync.scm (<rsync-configuration>): New record type.
(rsync-accounts, rsync-shepherd-service): New service extensions.
(rsync-service-type): New service type.
* gnu/tests/rsync.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
 doc/guix.texi          |  67 +++++++++++++++++++
 gnu/local.mk           |   2 +
 gnu/services/rsync.scm | 170 +++++++++++++++++++++++++++++++++++++++++++++++++
 gnu/tests/rsync.scm    | 136 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 375 insertions(+)
 create mode 100644 gnu/services/rsync.scm
 create mode 100644 gnu/tests/rsync.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 6b4b19d0c..35833ea68 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -15820,6 +15820,73 @@ Extra options will be passed to @code{git daemon}, 
please run
 @end table
 @end deftp
 
address@hidden Incremental file transfer
+
+The @code{(gnu services rsync)} module provides the following services:
+
address@hidden Rsync service
+
+You might want an rsync daemon if you have files that you want available
+so anyone (or just yourself) can download existing files or upload new
+files.
+
address@hidden {Scheme Variable} rsync-service-type
+This is the type for the @uref{https://rsync.samba.org} rsync daemon,
address@hidden record as in this example:
+
address@hidden
+(service rsync-service-type
+         (rsync-configuration))
address@hidden example
+
+See below for details about @code{rsync-configuration}.
address@hidden deffn
+
address@hidden {Data Type} rsync-configuration
+Data type representing the configuration for @code{rsync-service}.
+
address@hidden @asis
address@hidden @code{package} (default: @var{rsync})
address@hidden package to use.
+
address@hidden @code{port-number} (default: @code{873})
+TCP port on which @command{rsync} listens for incoming connections.  If
+port is less than @code{1024} @command{rsync} will be started as the
address@hidden user and group.
+
address@hidden @code{pid-file} (default: @code{"/var/run/rsyncd/rsyncd.pid"})
+Name of the file where @command{rsync} writes its PID.
+
address@hidden @code{lock-file} (default: @code{"/var/run/rsyncd/rsyncd.lock"})
+Name of the file where @command{rsync} writes its lock file.
+
address@hidden @code{log-file} (default: @code{"/var/log/rsyncd.log"})
+Name of the file where @command{rsync} writes its log file.
+
address@hidden @code{use-chroot?} (default: @var{#f})
+Whether to use chroot for @command{rsync} shared directory.
+
address@hidden @code{share-path} (default: @file{/srv/rsync})
+Location of the @command{rsync} shared directory.
+
address@hidden @code{share-comment} (default: @code{"Rsync share"})
+Comment of the @command{rsync} shared directory.
+
address@hidden @code{read-only?} (default: @var{#f})
+Read-write permissions to shared directory.
+
address@hidden @code{timeout} (default: @code{300})
+I/O timeout in seconds.
+
address@hidden @code{user} (default: @var{"rsyncd"})
+Owner of the @code{rsync} process.
+
address@hidden @code{group} (default: @var{"rsyncd"})
+Group of the @code{rsync} process.
+
address@hidden table
address@hidden deftp
+
 @node Setuid Programs
 @subsection Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index ff4777765..a1cace5ec 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -444,6 +444,7 @@ GNU_SYSTEM_MODULES =                                \
   %D%/services/shepherd.scm                    \
   %D%/services/herd.scm                                \
   %D%/services/pm.scm                          \
+  %D%/services/rsync.scm                       \
   %D%/services/sddm.scm                                \
   %D%/services/spice.scm                               \
   %D%/services/ssh.scm                         \
@@ -487,6 +488,7 @@ GNU_SYSTEM_MODULES =                                \
   %D%/tests/mail.scm                           \
   %D%/tests/messaging.scm                      \
   %D%/tests/networking.scm                     \
+  %D%/tests/rsync.scm                          \
   %D%/tests/ssh.scm                            \
   %D%/tests/web.scm
 
diff --git a/gnu/services/rsync.scm b/gnu/services/rsync.scm
new file mode 100644
index 000000000..769ad9c0c
--- /dev/null
+++ b/gnu/services/rsync.scm
@@ -0,0 +1,170 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Oleg Pykhalov <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services rsync)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages rsync)
+  #:use-module (gnu packages admin)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (ice-9 match)
+  #:export (rsync-configuration
+            rsync-configuration?
+            rsync-service-type))
+
+;;;; Commentary:
+;;;
+;;; This module implements a service that to run instance of Rsync,
+;;; files synchronization tool.
+;;;
+;;;; Code:
+
+(define-record-type* <rsync-configuration>
+  rsync-configuration make-rsync-configuration
+  rsync-configuration?
+  (package       rsync-configuration-package              ; package
+                 (default rsync))
+  (port-number   rsync-configuration-port-number          ; integer
+                 (default 873))
+  (pid-file      rsync-configuration-pid-file             ; string
+                 (default "/var/run/rsyncd/rsyncd.pid"))
+  (lock-file     rsync-configuration-lock-file            ; string
+                 (default "/var/run/rsyncd/rsyncd.lock"))
+  (log-file      rsync-configuration-log-file             ; string
+                 (default "/var/log/rsyncd.log"))
+  (use-chroot?   rsync-configuration-use-chroot?          ; boolean
+                 (default #f))
+  (share-path    rsync-configuration-share-path           ; string
+                 (default "/srv/rsyncd"))
+  (share-comment rsync-configuration-share-comment        ; string
+                 (default "Rsync share"))
+  (read-only?    rsync-configuration-read-only?           ; boolean
+                 (default #f))
+  (timeout       rsync-configuration-timeout              ; integer
+                 (default 300))
+  (user          rsync-configuration-user                 ; string
+                 (default "root"))
+  (group         rsync-configuration-group                ; string
+                 (default "root"))
+  (uid           rsync-configuration-uid
+                 (default #f))
+  (gid           rsync-configuration-gid
+                 (default #f)))
+
+(define (rsync-account config)
+  "Return the user accounts and user groups for CONFIG."
+  (let ((rsync-user  (rsync-configuration-user config))
+        (rsync-group (rsync-configuration-group config)))
+    (list (user-group (name rsync-group) (system? #t))
+          (user-account
+           (name rsync-user)
+           (system? #t)
+           (group rsync-group)
+           (comment "rsyncd privilege separation user")
+           (home-directory "/var/run/rsyncd")
+           (shell #~(string-append #$shadow "/sbin/nologin"))))))
+
+(define (rsync-activation config)
+  "Return the activation GEXP for CONFIG."
+  #~(begin
+      (use-modules (guix build utils))
+      (let ((share-directory #$(rsync-configuration-share-path config))
+            (user            #$(rsync-configuration-user config))
+            (group           #$(rsync-configuration-group config)))
+        (mkdir-p (dirname #$(rsync-configuration-pid-file config)))
+        (and=> share-directory mkdir-p)
+        (if (not (eq? user "root"))
+            (chown share-directory
+                   (passwd:uid (getpw user))
+                   (group:gid (getpw group)))))))
+
+(define (rsync-config-file config)
+  "Return the rsync configuration file corresponding to CONFIG."
+  (let ((port-number   (rsync-configuration-port-number   (pk 
'rsync-config-file-config config)))
+        (pid-file      (rsync-configuration-pid-file      config))
+        (lock-file     (rsync-configuration-lock-file     config))
+        (log-file      (rsync-configuration-log-file      config))
+        (use-chroot?   (rsync-configuration-use-chroot?   config))
+        (share-path    (rsync-configuration-share-path    config))
+        (share-comment (rsync-configuration-share-comment config))
+        (read-only?    (rsync-configuration-read-only?    config))
+        (timeout       (rsync-configuration-timeout       config))
+        (user          (rsync-configuration-user          config))
+        (group         (rsync-configuration-group         config))
+        (uid           (rsync-configuration-uid           config))
+        (gid           (rsync-configuration-gid           config)))
+    (if (not (eq? user "root"))
+        (begin
+          (if (<= port-number 1024)
+              (error (string-append "rsync-service: to run on port "
+                                    (number->string port-number)
+                                    ", user must be root.")))
+          (if use-chroot? (error (string-append
+                                  "rsync-service: to run in a chroot"
+                                  ", user must be root.")))
+          (if uid (error "rsync-service: to use uid, user must be root."))
+          (if gid (error "rsync-service: to use gid, user must be root."))))
+    (mixed-text-file
+     "rsync.conf"
+     "# Generated by 'rsync-service'.\n"
+     "pid file = " pid-file "\n"
+     "lock file = " lock-file "\n"
+     "log file = " log-file "\n"
+     "port = " (number->string port-number) "\n"
+     "use chroot = " (if use-chroot? "true" "false") "\n"
+     (if uid (string-append "uid = " uid "\n") "")
+     (if uid (string-append "gid = " gid "\n") "")
+     "\n"
+     "[files]\n"
+     "path = " share-path "\n"
+     "comment = " share-comment "\n"
+     "read only = " (if read-only? "true" "false") "\n"
+     "timeout = " (number->string timeout) "\n")))
+
+(define (rsync-shepherd-service config)
+  "Return a <shepherd-service> for rsync with CONFIG."
+  (let* ((rsync       (rsync-configuration-package config))
+         (pid-file    (rsync-configuration-pid-file config))
+         (port-number (rsync-configuration-port-number config))
+         (user        (rsync-configuration-user config))
+         (group       (rsync-configuration-group config)))
+    (list (shepherd-service
+           (provision '(rsync))
+           (documentation "Run rsync daemon.")
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$rsync "/bin/rsync")
+                           "--config" (pk 'shepherd-rsync-config-file 
#$(rsync-config-file config))
+                           "--daemon")
+                     #:pid-file (pk 'shepherd-pid-file #$pid-file)
+                     #:user (pk 'shepherd-user #$user)
+                     #:group (pk 'shepherd-group #$group)))
+           (stop #~(make-kill-destructor))))))
+
+(define rsync-service-type
+  (service-type
+   (name 'rsync)
+   (extensions
+    (list (service-extension shepherd-root-service-type rsync-shepherd-service)
+          (service-extension account-service-type rsync-account)
+          (service-extension activation-service-type rsync-activation)))
+   (default-value (rsync-configuration))))
diff --git a/gnu/tests/rsync.scm b/gnu/tests/rsync.scm
new file mode 100644
index 000000000..782c2355c
--- /dev/null
+++ b/gnu/tests/rsync.scm
@@ -0,0 +1,136 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Christopher Baines <address@hidden>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu tests rsync)
+  #:use-module (gnu packages rsync)
+  #:use-module (gnu tests)
+  #:use-module (gnu system)
+  #:use-module (gnu system file-systems)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system vm)
+  #:use-module (gnu services)
+  #:use-module (gnu services rsync)
+  #:use-module (gnu services networking)
+  #:use-module (guix gexp)
+  #:use-module (guix store)
+  #:export (%test-rsync-with-default-port
+            %test-rsync-with-port-2000))
+
+(define* (run-rsync-test rsync-os #:optional (rsync-port 581))
+  "Run tests in %RSYNC-OS, which has rsync running and listening on
+PORT."
+  (define os
+    (marionette-operating-system
+     rsync-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (port-forwardings '())))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-11) (srfi srfi-64)
+                       (gnu build marionette))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "rsync")
+
+          ;; Wait for rsync to be up and running.
+          (test-eq "service running"
+            'running!
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'rsync)
+                'running!)
+             marionette))
+
+          ;; Make sure the PID file is created.
+          (test-assert "PID file"
+            (marionette-eval
+             '(file-exists? "/var/run/rsyncd/rsyncd.pid")
+             marionette))
+
+          (test-assert "Test file copied to share"
+            (marionette-eval
+             '(begin
+                (call-with-output-file "/tmp/input"
+                  (lambda (port)
+                    (display "test-file-contents\n" port)))
+                (zero?
+                 (system* "rsync" "/tmp/input"
+                          (string-append "rsync://localhost:"
+                                         (number->string #$rsync-port)
+                                         "/files/input"))))
+             marionette))
+
+          (test-equal "Test file correctly received from share"
+            "test-file-contents"
+            (marionette-eval
+             '(begin
+                (use-modules (ice-9 rdelim))
+                (zero?
+                 (system* "rsync"
+                          (string-append "rsync://localhost:"
+                                         (number->string #$rsync-port)
+                                         "/files/input")
+                          "/tmp/output"))
+                (call-with-input-file "/tmp/output"
+                  (lambda (port)
+                    (read-line port))))
+             marionette))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "rsync-test" test))
+
+(define* (%rsync-os #:optional (rsync-port 581))
+  ;; Operating system under test.
+  (let ((base-os
+         (simple-operating-system
+          (dhcp-client-service)
+          (service rsync-service-type
+                   (rsync-configuration
+                    (port-number rsync-port))))))
+    (operating-system
+      (inherit base-os)
+      (packages (cons* rsync
+                       (operating-system-packages base-os))))))
+
+(define %test-rsync-with-default-port
+  (system-test
+   (name "rsync-with-default-port")
+   (description "Connect to a running RSYNC server.")
+   (value (run-rsync-test (%rsync-os)))))
+
+(define %test-rsync-with-port-2000
+  (let ((port 2000))
+    (system-test
+     (name "rsync-with-port-2000")
+     (description "Connect to a running RSYNC server.")
+     (value (run-rsync-test (%rsync-os port) port)))))
-- 
2.14.1