Re: Initial target(s) for libmob

[Carl Fredrik Hammar]

Hi,

On Sat, May 30, 2009 at 06:50:01AM +0200, olafBuddenhagen@gmx.net wrote:
> 
> > > ioctl()s are always specific to a particular device class, and thus
> > > the server(s) implementing (or proxying) it. It makes perfect sense
> > > for a server implementing a specific device, also to provide the
> > > ioctl wrappers necessary to access it.
> > 
> > I agree that ioctls are device specific, but not that they are
> > specific to the server.  If the client makes use of an ioctl, then it
> > must be aware of the device class as well.  A header must be provided
> > that defines macros for the ioctl numbers, it does not seem
> > unreasonable to also provide a module with the corresponding handlers
> > at the same time.
> 
> Actually, the header is normally provided by the kernel, isn't it? And
> thus, the entity implementing the drivers... So, my point stands :-)

Normally yes.  But we're in a slightly different position than normal
kernels.  In particular it is likely that that we want to supply the
header before we actually have an implementation in place, so that
applications compile.

> > Broadly speaking there are only two options; either the device
> > specifies it or a system service does.  The system service can be
> > quite mundane, as my original suggestion of simply loading a library
> > based on the device class name.  Note that the name would have to be
> > encoded into the ioctl number, or looked up in a file e.g.
> > /etc/ioctltab, otherwise glibc would need to be hacked in order to add
> > new device classes.  It could also be a more sophisticated system
> > server, possibly giving out handlers using libmob as in your
> > suggestion.
> 
> Err...

This is was my conclusion as well.  ;-)

> > Lets look at this from the perspective of a non-root user who wants to
> > provide a device of a new device class.  This is the sort of thing the
> > Hurd is all about.  ;-)
> 
> ...that's just what I was thinking when reading the paragraph above :-)
> 
> As I already said in the unionmount discussion (regarding the hidden
> nodes), we generally want to avoid centralized services, unless really
> necessary. When the thing can be negotiated between the server and the
> client directly, why introduce a central service? It only complicates
> stuff and limits possibilities.

Yes, I have begun to appreciate this property more and more.

> > One issue with encoding the handler module name in the ioctl is that
> > it requires changing the current numbering convention.
> 
> No idea what you mean :-(

Well the current numbering convention has a field for which device
class the ioctl belongs to.  It encodes a single letter, and my
idea was to simply load ``ioctl-$DEVICE_CLASS_LETTER''.  I concluded
that a single letter was pretty confining and that a change would be
necessary.

Though, for some reason I only considered it relevant if we chose to
encode the module name in the ioctl number.  But this is affects all
possible implementations.  Looking closer, it seems that the field
only allows for 16 (!) device classes (the letters f to v).  This seems
unacceptably low if we want to support any meaningful number of ioctl
classes.

(Most of the number encodes parameter type information, which isn't very
useful IMHO.)

Changing it would be painful, as all applications that currently
uses ioctls will need to be recompiled.  If this is hard to detect,
it might mean that *all* applications will need to be recompiled in
practice.

This is less of a problem if we support clashing ioctls.  Because then
we could support two numbering schemes during an interim period.

Looking through our current ioctls there are some clashes.  I'm unsure
how this is handled currently, especially since they deviate from
the standard numbering convention.  Perhaps they are just aliases.
Perhaps they don't have any parameters and thus don't need handlers.
Or perhaps they aren't handled at all...

> > Another interesting issue is ioctl clashes, i.e. that two distinct
> > ioctls share the same number but differ in semantics and possibly
> > parameters and their types.
> 
> Is that possible on other systems? Otherwise, the only case where I
> could see that happen, would be if we want to emulate ioctl()s from more
> than one other system...

Well, Linux has documented clashes.  This doesn't really affect us
however, since we can just define the ioctl to use a different number.
Unless we want ABI compatibility that is.  But I've been assuming that
we don't.

I guess it's possible that other systems define ioctl macros with the
same name in the same header that have different semantics.  Then we
would be forced to use the same number.  I don't think this is the case,
but who knows?

> > However, I also want to get a thesis out of this and I don't think
> > server loaded ioctl handlers would be enough on their own.
> 
> I think they actually might -- but I don't know the expectations there,
> so I can't really tell...

I was considering a thesis limited to ioctls, and I don't think it's enough
because code-loading would be the only interesting part.  But again, it
might work with a pretext that it's an initial use-case for a generic
framework which needs a more beefy object system and needs to pull
in dependencies.

> Thus, when talking about use cases, I was thinking about more specific
> things. Moving libstore to use the more generic framework, seems the
> only other realistic target here... We don't really have any other cases
> that employ translator stacking at any meaningful scale so far.

I was planning on implementing a proof of concept translator and client
using normal IO.  But more generic framework, e.g. not hard-coded for IO.

Porting libstore would be nice, but it isn't very meaningful in itself.
It would be more proper to wait until the framework is used elsewhere.
Perhaps ioctls would suffice in this, I'm not sure.  (I'm getting a
deja-vu, have we discussed this before?)

> > However, I would like to finish my thesis before the next opportunity
> > to hand it in for review, which is September 2.  So I'm mostly worried
> > that implementing ioctls will keep me from completing it on time.
> 
> It seems to me that the ioctl() use case should be almost trivial
> compared to the optimization one. It shouldn't require much aside from
> the common components of the framework I would think, which you would
> need for the latter anyways...

I'm not worried about the framework, but making use of it in glibc.
I'm not familiar with glibc development.  It's only quite recently that
I've been able to browse the code without feeling hopelessly lost.  :-)

I would also need to get a better understanding of the current
implementation.  Which is daunting considering that ioctl itself is a
massive 300 LOC function, which calls support functions and handlers.

Also it just struck me that the ioctls would drag the whole framework into
glibc itself (or most of it anyway).  Not sure how I feel about that.  :-(

(Sometimes I wish glibc was somehow more modular, so that you could
bring in functionality without explicit linking.  glibc seems like a
bottle-neck for decentralization.  Hmm... perhaps there is a use-case
somewhere around here.)

Regards,
  Fredrik