cgroupfs, /hurd/proc and subhurds

[Justus Winter]

Hi :)

this mail discusses my recent attempts of creating a cgroupfs, related
problems and issues I encountered so far.

Problem statement
=================

Linux has this feature called cgroups. It groups processes (threads)
together in groups, furthermore so called controllers can be used to
restrict the use of various resources (like cpu time, memory) on a
per-group basis. A notable feature of cgroups is that a process cannot
escape its group, and any children of a process are born into the same
group as the parent.

In order to create a cgroupfs on Hurd, one has to make the same
guarantee. Currently the parental relationship of processes is a
Hurd-only concept, established by the parent process calling
proc_child. This is not robust enough, as a process can create a new
process using task_create and not claim ownership of that process.

cgroup documentation:

https://www.kernel.org/doc/Documentation/cgroups/cgroups.txt

Related problems
----------------

The proc server has to be privileged (needs the host-priv port) so
that it can query the kernel for new processes (see proc/mgt.c
(add_tasks)). It does that mainly when a process triggers a lookup
like proc_task2proc for newly created tasks. Ironically then the task
port for the new process is already known, still add_tasks requests
*all* task ports from the kernel, only to mach_port_deallocate all the
ones it already knows and to add new tasks (in my test it was always
just one, the newly created task of which the task port is already
known). I believe that this is done in the lookup mainly to do this
periodically, so that any task will eventually get noticed (but see
below). I do not know how expensive this is in practice, but it seems
very wasteful and unnecessary to me.

So there are three related issues:

1. Non-root users cannot start sub-hurds:
   https://savannah.gnu.org/bugs/?17341

2. Due to the process of discovering all task ports, any sub-hurd gets
   a handle of any task running on the system, so root users inside a
   sub-hurd can interfere with the operation of the parent hurd, which
   is undesirable from an isolation point of view. This is also
   related to 1. from a security point of view.

3. add_tasks is unnecessary and potentially wasteful.

/hurd/proc is a dark corner indeed
----------------------------------

The routine description of proc_child looks harmless enough:

/* Declare that a task is a child of the caller.  The task's state
   will then inherit from the caller.  This call can be made only once
   per task. */
routine proc_child (
        process: process_t;
        child: task_t);

But dragons are lurking here. sysdeps/mach/hurd/fork.c explains this
best:

/* Register the child with the proc server.  It is important that
   this be that last thing we do before starting the child thread
   running.  Once proc_child has been done for the task, it appears
   as a POSIX.1 process.  Any errors we get must be detected before
   this point, and the child must have a message port so it responds
   to POSIX.1 signals.  */
if (err = __USEPORT (PROC, __proc_child (port, newtask)))
  LOSE;

So proc_child not only declares that the newly created process is ones
task, but it also indicates that the process is all set up and ready
to receive POSIX signals. Surely enough all hell broke loose when I
tried to use my shiny new notification system (see below) to supply
the parental relations instead of waiting for someone to call
proc_child.

Proposed solution
=================

I propose a notification based system to fix all of the above issues:

1. The topmost /hurd/proc server registers for notifications at the
   kernel. The notifications are sent when a new task is created and
   carry the task ports of both the parent and the newly created task.
   Only one process can register for this notifications and it has to
   have the host_priv port to authenticate itself. This makes the
   kernel implementation quite tiny and unobtrusive.

2. Anyone can register for process change notifications at the proc
   server. These notifications carry the PID and PPID of newly created
   processes (and ones that died) and allow cgroupfs to implement the
   cgroup semantics. This information could also be obtained by
   polling the proc server and diffing the results, so it should not
   be necessary to restrict the usage of this interface.

3. A proc server running in a sub-hurd can register at the topmost
   proc server for new task notifications, like the topmost proc
   server registers with the kernel. The proc servers are just a
   little bit sub-hurd aware, and because of the robust parental
   relationship of the tasks (*not* processes) provided by the kernel
   it can track which sub-hurd a task belongs to and notify the
   appropriate proc server.

Implementation
==============

I've created a proof of concept implementation for points 1. and
2. I'll send it as follow-ups to this mail. It contains:

* A general purpose notification library libhurdnotify.
* A port of /hurd/init to libhurdnotify.
* The new process notifications.
* New task notifications in gnumach, the proc server registers for
  those and they arrive, though nothing useful is done with them atm.

The cgroupfs repository can be found here:
http://darnassus.sceen.net/gitweb/teythoon/cgroupfs.git/

The state of cgroupfs is described in my last blog post:
https://teythoon.cryptobitch.de/posts/cgroupfs-is-as-cgroupy-as-it-gets/

I appreciate your input,
Justus