Starting work by extending ChacmOS's memsvr

[Farid Hajji]

Everyone interested in the l4-hurd port should definitively
take a [second] look at ChacmOS:

  http://l4ka.sourceforge.net/download/ChacmOS-src-RC1.tgz

This is a small multi server OS running on top of L4, written by
students for educational purposes. ChacmOS lacks driver support
(putc() writes directly to screen memory at 0xb8000 and
getc() gets its input directly from the keyboard).

Nonetheless, for Hurd hackers, ChacmOS is interesting for the
following reasons:
  * It exercises the L4 C-API, so we can learn from it.
  * It uses multiple tasks than can RPC each other, somewhat
    like in the Hurd.
  * It's small enough to be studied in a reasonable time-frame!

ChacmOS's booter starts a few tasks that we could easily
extend to suit our needs:
  * memsvr: This primitive (physical) memory server could
            be revamped to provide full pager functionality
            (including logical memory addresses) and vm_*()
            mach-like semantics. [see below]
  * namesvr: The Hurd uses a distributed mechanism for the
             nameserver. But namesvr resembles the sname[d]
             a.k.a. bootstrapping-nameserver of Mach 3.0,
             so we may need as well, at least until we have
             the Hurd root filesystem and glibc(!) in place.
  * ext2fs/filesvr: This is actually a VFS-light implementation
                    of a [currently memory-based] filesystem.

Interesting about ChacmOS is that it uses IDL4 to generate
the real RPC stubs out of a .idl4 spec. This is exactly what
we do in the Hurd with MIG and the .defs files. The current
sources of ChacmOS provide the .idl files and the generated
stubs as well, because IDL4 is not yet publicly released.

So, what can we do right now? I'd suggest that we use ChacmOS
as a starting environment (this way, we make sure that no
hidden depencendies creep in) and that we incrementally
add the components/tasks and libraries that are needed to
provide at least a small subset of Mach. We need vm_*()
and some dev_t at the very least [The issues of port and
port rights must still be discussed later]. So to get vm_*(),
we need the pager mentioned in the TODO list, together with
the vm_*() client library that RPC this pager. To get dev_t
like in Mach, we need a few native L4 devices, or the framework
that was mentioned earlier on this list.

Now for the pager: memsvr seems like a good template to start with.
The IDL of memsvr is certainly inadequate here and would need to
be replaced by something more sensible. Two approches come to mind:

  * We can keep memsvr's interface as-is, and let every os-personality
    use its own pager that draws memory from memsvr on a FCFS basis.
    -> This would allow us to run L4Linux, [L4BSD?], Mach-emulation
       or whatever simultaneously, but we won't have a global memory
       allocator here. Then again, why use memsvr instead of sigma0
       at all?

  * We drop memsvr's interface in favor of a generic interface a la
    Mach's VM which is reasonably generic and which is used either
    directly or indirectly in most current kernels (including Mach,
    of course ;-)). 
    -> OS servers' own memory management, be it VM, UVM or whatever,
       would need to be either overridden or somewhat connected to
       the global VM pager. For the Hurd, this is not an issue, because
       we use Mach directly; for L4Linux or other OS-Servers that
       draw memory from sigma0, this is not so easy.

The advantage of the second approach (a revamped memsvr's VM inteface,
a.k.a. a global pager for all OS personalities, including the Hurd) is
that we can use the locality of reference of clients running on
multiple parallel OS-personalities at the same time: Consider a
situation with parallel [modified] L4Linux (just an example) and the
Hurd, where one client runs on top of L4Linux and a few clients run on
the Hurd. All clients would draw memory from the global pager, which
would ensure that the union of the working sets of all active
processes/tasks is resident/mapped.

Could we start by defining a IDL for a generic L4 pager as first step?
I'd suggest that we revisit Mach's external pager interface first
(that's the protocol between Mach itself and its pager: we don't need
this explicitely, but its intersting to know about it anyway), and
at the vm_allocate(), vm_dealloate(), vm_map(), vm_unmap(), but also
vm_remap() syscalls. This would naturally lead to an IDL definition
of the pager.

Of course, we can't start hacking now, as long as IDL4 is not publicly
released. One work-around would be to try DICE, the other work-around
would be to do the IPC manually, but I hope that that the latter won't
be necessary ;-).

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | address@hidden
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
One OS To Rule Them All And In The Darkness Bind Them... --Bill Gates.