Critique of libchannel's design

[Carl Fredrik Hammar]

Hello,

I haven't been able to put any real work into libchannel for a while,
but I have been brooding and scribbled down some ideas.  So before my
studies resume, I'd like to consolidate my thoughts into a text and
hopefully get some feedback.  This will done in a series of mails,
this first one will be a critique of my previous design and some
conclusions drawn from it.

These mails will also be an indirect response to Olaf's last messages
in the previous discussion about libchannel's design [1].  I'd like to
apologize for the lateness of this reply, and thank Olaf for the mail
which I agree with and has inspired the current direction of the
design.

The current libchannel tries to form a abstraction for character
devices ranging from simple input devices to not so simple network
interfaces.  The major feature being that the implementation can then
be transferred from a server to a client and then run directly.  This
is particularly useful in stacks of translators, where each layer
provides additional functionality to the layer beneath it.  These
stacks can potentially be very deep with each layer communicating
through expensive IPC, but if all layers are implemented through
libchannel there is only IPC between client and the top layer.

To implement a channel one writes a module which implements a certain
interface.  When a client requests a channel it's data is encoded and
transferred along with a tag that specifies its implementing module,
libchannel checks that the module is loaded and loads it if it is not.
The module decodes the data and returns a fully fledged channel.

The channelio translator is also provided as the main means with which
to use channels, its main task is to parse a channel description and
loads necessary modules to create the channel and then simply
translate io and fs RPCs to operations on that channel.

So whats the problem with the current design?

The core of the problem is that the channel abstraction turned out to
be much more complicated than simply passing a stream bytes around.
At first it seemed like a subset of stores that only deals with simple
io, but only few assumptions can be made on the semantics of the
stream of bytes that pass through the channel.  Where can it be split
into chunks?  How much can be buffered?  How do different sessions
interact?  These are all questions which are open in case of general
channels, which complicates the relation between libchannel and
channelio.

But the same questions have straight forward answers when it comes to
stores, in this sense stores are a subset of channels.  This also
applies to other special cases such as simple character streams,
audio, and the various layers in a network stack.

In response to this the current libchannel and channelio attempted to
allow the construction of (almost) arbitrary translators, but from
within a very rigid framework.  It comes fairly close to this goal,
but if it's later discovered that the framework can't deal with some
unanticipated case, it must be rewritten which could break existing
modules.

Part of the solution is to enable translators to use libchannel to
implement a interface that, essentially, allows it to be transferred
and run inside another translator.  That is, instead of a single
special channelio translator there is one per module.  This allows
much more flexibility, and as a bonus modifying a normal translator to
implement channels will be transparent for users.

The rest of the solution is to generalize the channel concept in order
to minimize libchannel, so that it *only* provides a means of
transferring the channels implementation from one translator to
another, without assuming any specific semantics.  To simplify
implementation of the translators themselves, libtrivfs style
libraries can be provided, e.g. libaudio, libnet, and possibly a new
libstore. 

This concludes my first mail on the subject.  In my next mail I will
state and motivate the minimal requirements of a new general channel
concept and also some of its basic properties.

Regards.
  Fredrik

[1] ``A Vison Regarding Channels (was: Re: Channel sessions)'' -
    http://lists.gnu.org/archive/html/bug-hurd/2007-09/msg00004.html