Re: Porting the Hurd to L4 (glibc dependencies, dropping glibc?)

[Farid Hajji]

Roland,

> > need to reply like this. Basically, you suggest nothing less than to shut
> > up and do our own non-approved stuff without asking for feedback from the
> > list. This is asking for a split in development :-((( Sad perspectives...,
> > but splits are necessary sometimes.
> I didn't say "shut up", I said "less talking and more hacking".  The
> talking to hacking ratio is way too high around here.  I'll have more to
> say and more interest in reading long email exchanges if there is active
> work going on in some concrete direction or other.  Don't worry about
> stepping on my toes; I don't care.  I'd just like not to have what little
> brain time the Hurd gets from the world too bogged down in speculative
> discussions when there is so much real code that we could all be writing.
I'm sorry, but I clearly over-reacted yesterday. Of course, you're right
about the S/N (a.k.a hacking/talk) ratio on the lists and more hacking would
do all of us well. I was just so terribly angry with glibc not compiling on
my FreeBSD system that I was about to give it up completely.

Not being able to use glibc right now does have its advantages though, and
after examining the code in more detail, I'd like to elaborate a bit on
what I'm trying to achieve right now. But be forwarned: A port of the Hurd
is not a simple set of diffs against the current source tree! A tremendous
amount of work (and modifications) seems necessary because of the Hurd being
extremely intertwined with mach (and to a smaller degree with glibc). To get
down to the details, here are some hurd-specific questions:

1. I'm trying to put the Hurd on top of a 'vk' (virtual kernel) API
that will have to provide a minimal set of semantics required of
microkernels.  Ultimately, there would be many vk libraries to choose
from, e.g.: vk-mach, vk-l4, vk-linux, vk-freebsd, vk-solaris, ... Each
vk would implement the ukernel semantics (vk-API) in a different way, using
what's available underside. vk-mach would map tasks, threads and ports
to the familiar mach counterparts, vk-l4 would do something similar as
far as possible and vk-{guest-os} would simulate ukernels with real
Unix processes and pthreads. All vk-[*] would have the same public
interface, as far as the Hurd is concerned.

  -> What can be considered a minimal set of ukernel semantics?
     Something very small that we could probably agree upon would
     be basically what L4 provides: tasks, threads (in limited number),
     and a synchronouus [opaque-]message passing interface. Add to this
     vm_address_t (as parts of tasks) and some exception handling (and
     notification) mechanism to get at the devices and page faults.
     Simple entry points to libvk would be vk_port_t descriptors (typedef
     int vk_port_t;) that could supersede most of mach_port_t ports in
     the Hurd.

     IMHO, we should brainstorm and then agree on such a minimal interface
     that will have to be provided by those vk-[*] libraries. The smaller
     the required set of semantics, the easier it would be to port libvk
     to yet another ukernel or guest-os.

     One important issue in designing a public interface to libvk would
     be to simplify this interface to the max (or to the min, if you
     so prefer) for efficiency reasons. I'm thinking here especially
     of an L4-style simplification of the ipc, and it would be good
     to be able to implement libvk-l4 by simply __inline__(ing) some
     L4 ipc calls for the (mostly used) generic vk_ipc_*() calls. Only
     then can we hope to achieve a performance boost like what L4 promises
     [to non L4-initiated: L4 tries to pass as much information in registers
      and if not possible, it tries to squeeze as much in the TLB of the
      processor to avoid cash misses].

I'd suggest to keep libvk totally separate from glibc, so that it can be
easily developed on guest-os(es), on L4 (e.g. L4-ka) build environments
and probably also with [oskit-]gnumach (iff gnumach compiles w/o glibc).
libvk-[*] could then be dropped into the hurd simply by linking everyting
against it.

2. Next comes libports, which is used in almost every hurd server and
   hurd library. I'm not sure yet how libports should fit in the context
   of a libvk-based Hurd and I need guidance from more experienced folks
   on this right now.

   libports tries to catch a lot of mach specialities in a somewhat higher
   level abstraction, mainly defined in libports/ports.h. Here comes to
   mind port_info, port_bucket, port_class, rpc_info, rpc_notify, ....
   as well as a lot of helper functions ports_*() spread all over the
   libports/* files.

   All this seems incredibly complex and is extremely mach-centric.
   I'm wondering what this is really good for and if it is needed once
   a switch to libvk is done. libports seems somehow to fit below
   (or inside) libvk-mach but has higher ramifications too and cannot
   simply be put aside.

   I'll try to slowly chew myself into libports but if you could
   enlighten me on the purpose of it, it would be much faster.

   -> If we don't use mach anymore (except via libvk-mach), what parts
      of libports would be _really_ needed (architecturally)? This seems
      like one of the most important porting decisions right now.

   -> At first sight (I didn't analyze the code in greater detail yet, so
      I may be plain wrong here) libports _seems_ to hinder a lean
      L4-style ipc thus adding a lot of ipc overhead which happens so
      slow the Hurd down anyway. I'm I right to guess that libports
      is mainly a hack to get an interface to the complex intricacies
      of mach? How much of libports _is_ actually hurd, and how much
      is mainly struggling with mach? I'd _really_ like to drop libports
      on non-mach libvk hurd's if at all possible, or at least replace
      it with a minimum non-mach subset.

3. Once the libports issue is resolved, it's time to look at the ipc
   protocol between the Hurd servers (and their libraries). The protocol
   is mainly defined in hurd/*.defs, which is IMHO [one of] the most
   interesting part of the Hurd.

   Of course, .defs are transformed by mig into mach stubs and a port
   of mig would then generate libvk stubs instead. Porting mig was
   already discussed here and it was even suggested to have a look
   at IDL compilers or somesuch. Wether this is overkill or not, it
   is finally irrelevant. Let's assume that we port mig to generate
   libvk stubs in a fairly straightforward manner [it must still be
   checked wether this is possible. It all depends on what the libvk
   API would look like].

   Now I'm wondering how to map such mach stubs (which ultimately all
   end up sending or receiving a message from a mach_port_t) cleanly to
   a still-to-be-defined libvk ipc-API. If we decide to abandon the
   asynch IPC model of mach in favor of the more efficient sync ipc model
   of L4, I'm wondering what we should do about the routine/simpleroutine
   pairs often found here.

   One example: the auth server defines a routine auth_user_authenticate()
   which inherently returns a value (newport as a mach port) and in addition
   to this a simpleroutine auth_user_authenticate_reply() which sends a
   non-overlapping set of data too. I didn't yet figure out why both
   interfaces are needed and how they are exactly used (probably deep
   inside glibc/hurd???).

   Completing the handshake between auth (or btw any other server) and
   its client(s) need not be asynchronous in nature. Basically, a thread
   in the client could block on ipc while waiting for a reply from the
   server and the server could provide a few service threads to receive
   sync messages and service them entierely [receiving on a port-set
   in libvk-mach, select() or pthreads in libvk-{guestos} and ??? with
   L4 lthreads in libvk-l4]. Basically, we need to rethink the way
   how hurd servers are being ipc-ed to in general.

   -> Has anyone spent some time thinking about this? Please share your
      thoughts with us.

4. Other Hurd libraries, bootstrap process and a lot more issues I didn't
   considered yet. I'd prefer to gradually proceed and concentrate on a
   few well defined goals.

   -> If there is anything fundamental there which contradics with 1., 2.,
      or 3., please let me know ahead of time.

As already said, because I can't compile glibc on my FreeBSD box, I'll
try to modularize the work by compiling one bit at a time. Starting with
libvk-freebsd, other parts like the ominous libports and certain servers
could be compiled against the guest-os libc if they are somehow using
POSIX-ish C API. Even without any libvk-* concept yet, I was able to
compile libihash [spin_lock() replaced by pthread_mutex_lock() of
FreeBSD's libc] and libidvec [libshouldbeinlibc/idvec* files, having
to tweak a little without shadow.h and the non-reentrant getpwuid()s],
so there's hope after all ;-)

Any comments, suggestions and help appreciated.

Thanks,

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | farid.hajji@ob.kamp.net
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
Murphy's Law fails only when you try to demonstrate it, and thus succeeds.