Re: GSoC libchannel project draft

[Carl Fredrik Hammar]

I have attached the second revision of my proposal which I will send
in tomorrow.  It addresses the concerns made by Richard.  I decided to
not to make an attempt at a sketch of handling ioctls, but handling
them are included as a part of the project's goals.

I also made a commitment of having a complete design draft ready by
the midterm evaluation.  It's a little vague, but it's difficult to do
hard commitments this early, do you guys think I should commit to
more?

Other comments and suggestions are appreciated as always. :-)

Fredrik

Title   GNU Hurd, libchannel
Student Fredrik Hammar


Abstract

The GNU Hurd has long needed a library that abstracts the idea of a
character device to ease the clean and efficient implementation of the
servers that will implement these.  It has been recognized as needed
for a long time and some tasks have been stalled because of it's lack.

Not only could such a library enable quicker and cleaner
implementations of many translators that directly corresponds to
kernel implemented devices, such as keyboards and mice.  It would also
allow efficient implementations of character devices that are layered
on top of other streams.

The goal of this project is to design, implement and document
libchannel, that will be a library that answers the needs described
above.


Rationale

There are many cases where layering a character devices on top of
another character device is desirable.  For instance flexibility a
translator that balance the network load over several networks without
changing pfinet's implementation.  A schematic of how this would look
like is shown below.

eth0
   |-tun0-pfinet
eth1

Where `tun0' is the balancing translator that uses the NICs `eth0' and
`eth1'.

There is a heavy performance penalty for doing something like this,
instead of keeping all the processing in the pfinet server, the data
needs to be first passed through another process.  The communication
overhead is to high compared to the work needed for balancing.  This
example was taken from one of the discussions in [1].

Another problem often associated with character devices is ioctls that
takes pointers to arguments stored in structures.  This is handled by
the kernel in UNIX like systems, but in the Hurd hooks in libc for
each device with ioctl are required to handle them.  This makes
implementing such devices a painful experience.


Proposal

Designing and implementing of `libchannel', a library for the GNU Hurd
that eases the clean and efficient implementation of character devices
in the manner described in the previous section.

The need for such a library has long been recognized by the Hurd
community (discussions in [1] took place 2000) and a library for doing
the same thing for block devices, called `libstore' (see [2]), has
already been developed.

Since libchannel is conceptually very similar to libstore, the design
of libchannel will be heavily modeled after libstore.  Not only is it
convenient for me to have such a model to work after, but this analogy
is important for the ease of use of the library.  Having differences
other than the ones stemming from natural differences between blocks
and streams could only bring confusion to the future users of these
two sibling libraries.

The basic idea is that instead of sending the stream of data over IPC,
the channel itself can be sent over IPC and then used directly by the
receiving party.  Thus indirection is avoided.

Similar to libstore, libchannel will define a type that will be a
general channel abstraction. functions working against instances of
this and the means for implementing classes that will provide the
functionality of a channel.  Unlike a store however channels will also
provide a clean way of handling any ioctls.

The library will also provide a number of standard classes that are
deemed generally useful. For instance, `device' and `file' channels
that respectively implements a thin wrapper around a device driver in
the kernel and a filesystem node.  These would form a base layer.
Channels layered on top of these can filter the stream or aggregate a
number of channels by interleaving them.

Since there is a need to wrap channels around kernel devices, the
library can also be used to cleanly implement character devices for
these drivers with ease.  This leads to a concentration of the kernel
communication that might be useful in the future, since there has been
proposals to switch from the GNU Mach kernel to a more modern and
capable one, see [3].


Road-map

Below follows a rough sketch on how I plan to tackle this project.

* Study libstore
** Identify and examine classes that could be useful to implement.
** Examine the implementation of IPC in detail.
** Examine how security is achieved.

* Study storeio
** Examine how it uses libstore.

Studies will follow to any dependency of above items if necessary,
such as GNU Mach or libtrivfs.

These preparations might be performed during the interim period as an
exercise to bond with the Hurd community.  As a side-effect the
documentation of studied items might be improved.

* Write a design draft.
** Core functionality.
*** Channel abstraction.
*** ioctls.
*** IPC.
** Classes.
*** Identify classes that are generally useful.
** Security, identify security considerations.

My goal is to have a complete design draft done by the midterm
evaluation.  Of course changes will be made as necessary; should
additional insights surface during implementation.  This doesn't mean
that some of the implementation won't be written before this.

* Implementation.
** Core functionality.
*** Channel abstraction.
*** ioctls.
*** IPC.
** Classes.
*** Base classes, e.g. `device' and `file'.
*** Converting and aggregating classes.

* Documentation.

Documentation can mostly be done in parallel with the implementation.


Other ideas

The similarities between libchannel and libstore might be more than
just conceptual.  The libraries might be able to share code through a
third library or it might even be possible to merge them into a single
library.  This could enforce the analogy between them.  I don't think
I dare to commit to alter libstore as well during the course of this
project, but effort will be made in keeping subsystems compatible.


Hopes and dreams

The development of some current and future planned servers of the GNU
Hurd are stalled in wait for libchannel, such as [4].  My hope is that
this project will prove very useful in implementing these and thus
further the ambitions of the GNU Hurd project and freedom for software
users in general.  I to am directly affected by this and can't access
the internet through the GNU Hurd due to PPPoE implementation being
stalled.  Thus I have to use GNU/Linux instead, which is also free of
course, but doesn't offer the flexibility that could be possible
through the GNU Hurd.

This project would also give me much wanted experience in working on
free software development and I hope to be able to use it to
contribute to the Hurd and other free software in the future.


About myself

I was born 1986 in Sweden where I still reside.  I've been interested
in computers since I was a little kid playing around with a Commodore
64 and my interest in free software developed 6 or so years ago, being
around 15 when I first installed GNU/Linux.  Though I have been
interested I have not made any substantial contribution to free
software yet, but have always wanted to.  My interest in the Hurd
started about a year ago, though I happily used it for a while I was
not able to get a internet connection working, thus limiting its
usability for me.

I'm currently at my second year of computer-science studies.  I
currently have no merits in the field outside of academic ones aside
for some small private projects and hope this project will provide me
with some.  My main interests in the field are OSes, compilers and
programming language theory.  Although I have not been able to study
them formally yet.

I am fluent in C, Java, Lisp and Python, but also have a good
understanding of Haskell, SML, C++ and assembly.  I'm also familiar
with the following tools that might be relevant; emacs, autotools,
make and gcc.

My merits might be a bit lacking, but in researching and writing this
proposal I have become confident that I will be able to preform this
task.


References

[1] Mailing list discussion regarding libchannel - 
    http://savannah.gnu.org/task/download.php?file_id=9801

[2] Documentation for libstore -
    http://www.gnu.org/software/hurd/doc/hurd_8.html#SEC75

[3] Next Generation Hurd page on the Hurd Wiki - 
    http://hurd.gnufans.org/bin/view/Hurd/NextHurd

[4] Reimplement pfinet -
    http://savannah.gnu.org/task/?5469