Re: Initial target(s) for libmob

[olafBuddenhagen]

Hi,

On Fri, Jun 05, 2009 at 12:01:54PM +0200, Carl Fredrik Hammar wrote:
> On Sat, May 30, 2009 at 06:50:01AM +0200, olafBuddenhagen@gmx.net
> wrote:

> > > > ioctl()s are always specific to a particular device class, and
> > > > thus the server(s) implementing (or proxying) it. It makes
> > > > perfect sense for a server implementing a specific device, also
> > > > to provide the ioctl wrappers necessary to access it.
> > > 
> > > I agree that ioctls are device specific, but not that they are
> > > specific to the server.  If the client makes use of an ioctl, then
> > > it must be aware of the device class as well.  A header must be
> > > provided that defines macros for the ioctl numbers, it does not
> > > seem unreasonable to also provide a module with the corresponding
> > > handlers at the same time.
> > 
> > Actually, the header is normally provided by the kernel, isn't it?
> > And thus, the entity implementing the drivers... So, my point stands
> > :-)
> 
> Normally yes.  But we're in a slightly different position than normal
> kernels.  In particular it is likely that that we want to supply the
> header before we actually have an implementation in place, so that
> applications compile.

OK, I see your point...

> > > One issue with encoding the handler module name in the ioctl is
> > > that it requires changing the current numbering convention.
> > 
> > No idea what you mean :-(
> 
> Well the current numbering convention has a field for which device
> class the ioctl belongs to.  It encodes a single letter, and my idea
> was to simply load ``ioctl-$DEVICE_CLASS_LETTER''.  I concluded that a
> single letter was pretty confining and that a change would be
> necessary.
> 
> Though, for some reason I only considered it relevant if we chose to
> encode the module name in the ioctl number.  But this is affects all
> possible implementations.  Looking closer, it seems that the field
> only allows for 16 (!) device classes (the letters f to v).  This
> seems unacceptably low if we want to support any meaningful number of
> ioctl classes.
> 
> (Most of the number encodes parameter type information, which isn't
> very useful IMHO.)

AIUI, the point of the parameter type encoding is allowing most ioctl()s
to be defined by a header file alone, without the need for any specific
wrappers in libc. It is not relevant for explicit ioctl() wrappers --
the wrappers are necessary precisely for the cases where the parameters
can *not* be expressed with the standard encoding!

As for the class (group) letters, it seems to me that the purpose is to
support the _SIO{,R,W,RW} macros for the most important classes used in
BSD. (And probably other systems -- don't know how portable these macros
are.)

Why we want to support these, is beyond me however. I would think that
invocations of these macros are always hidden in system-specific header
files -- but perhaps some programs invoke them directly for whatever
reason?...

Aside from the _SIO macros, the group/command distinction seems
meaningless. The wrapper handling simply uses a linear 13 (4+7) bit
namespace. It should be perfectly possible to load wrapper modules based
on that...

> > > Another interesting issue is ioctl clashes, i.e. that two distinct
> > > ioctls share the same number but differ in semantics and possibly
> > > parameters and their types.
> > 
> > Is that possible on other systems? Otherwise, the only case where I
> > could see that happen, would be if we want to emulate ioctl()s from
> > more than one other system...
> 
> Well, Linux has documented clashes.  This doesn't really affect us
> however, since we can just define the ioctl to use a different number.

Actually, if we want to keep _SIO compatibility, we can't just
arbitrarily redefine them.

So, if there are indeed clashes between two ioctl()s that both need
explicit wrapping, handling it per device is the only possible approach.
Effectively, that's what monolithic kernels do I guess.

As I already stated, IMHO that's the most natural way to handle it.

> > Thus, when talking about use cases, I was thinking about more
> > specific things. Moving libstore to use the more generic framework,
> > seems the only other realistic target here... We don't really have
> > any other cases that employ translator stacking at any meaningful
> > scale so far.
> 
> I was planning on implementing a proof of concept translator and
> client using normal IO.  But more generic framework, e.g. not
> hard-coded for IO.
> 
> Porting libstore would be nice, but it isn't very meaningful in
> itself. It would be more proper to wait until the framework is used
> elsewhere.

I don't understand why you consider libstore less meaningful than some
fabricated example...

> > It seems to me that the ioctl() use case should be almost trivial
> > compared to the optimization one. It shouldn't require much aside
> > from the common components of the framework I would think, which you
> > would need for the latter anyways...
> 
> I'm not worried about the framework, but making use of it in glibc.
> I'm not familiar with glibc development.  It's only quite recently
> that I've been able to browse the code without feeling hopelessly
> lost.  :-)

I see.

> I would also need to get a better understanding of the current
> implementation.  Which is daunting considering that ioctl itself is a
> massive 300 LOC function, which calls support functions and handlers.

Actually, now I looked at the code, I find it surprisingly simple.

What this "massive 300 LOC function" does, is simply the marshalling
normally done by MiG, except that it is done according to the type
specification supplied as part of the request, instead of a hard-coded
argument list like in normal MiG-generated stubs. Once you understand
the idea of the type encoding, it's rather straightforward.

The call out to explicit wrappers is rather trivial as well.

Even the macro magic for the standard type encoding, is not nearly as
crazy as I expected -- no idea why it got that reputation...

> Also it just struck me that the ioctls would drag the whole framework
> into glibc itself (or most of it anyway).  Not sure how I feel about
> that.  :-(
> 
> (Sometimes I wish glibc was somehow more modular, so that you could
> bring in functionality without explicit linking.  glibc seems like a
> bottle-neck for decentralization.  Hmm... perhaps there is a use-case
> somewhere around here.)

Yeah, I already suggested splitting out the Hurd-specific system call
wrappers from the rest of glibc, so things like this could be easily
modified. However, none of the people I talked to seemed interested at
all :-(

-antrik-