Re: Namespace-based translator selection; project details

[olafBuddenhagen]

Hi,

On Fri, May 23, 2008 at 06:50:47PM +0300, Sergiu Ivanov wrote:
> On Fri, May 23, 2008 at 1:07 PM, <olafBuddenhagen@gmx.net> wrote:

> The idea is much clearer for me now :-) As far as I can understand,
> the namespace-based translator selection will _not_ modify the
> filesystem permanently. Instead, it shall allow dynamic adding and
> removal of translators.

It doesn't in fact modify the underlying filesytem at all. Rather, it
proxies it, providing different views depending on the name suffix. If
some program does an open() on foo,,gunzip for example, the proxy will
start a gunzip translator, but not attach it to the actual foo node, but
rather to a virtual node provided by the proxy; and then perform the
actual open() on that virtual node, handing back the result to the
requesting program.

> Frankly speaking, this idea looks very beautiful to me, but also I
> feel quite at a loss as to the means by which the implementation of
> such functionality will be done :-(  I will dig as deep as I will able
> to to obtain the required knowledge.

As I already suggested, at least for the prototype, I believe it can be
implemented as a proxy which mirrors the underlying file system, and
does it's magic whenever a virtual file with the special suffix is
accessed.

I also already suggested studying unionfs and/or firmlink as examples of
translators that do some kinds of namespace proxying.

> > Layering on top should actually be the default action. Any static
> > translators present should only be ignored if explicitely requested.
> >
> 
> Could the user request the removal of a static translator before
> adding another one using the following syntax: 'file1,,-u,+x' ?

Yes, except that the '+' is not necessary, as it's the default action
anyways :-)

Also, I wouldn't call it "removal", as the translator actually stays in
place -- it's just ignored, and the underlying node is opened instead.

Thinking of this, I'm not sure whether in the case of a translator
stack, it's actually possible to ignore just some of the layers...
Perhaps it's only all or none. That would introduce some questions how
to handle translator stacks...

> > I want to avoid any clashes with normal file names as well as
> > possible.
[...]
> > I thought about it quite a bit, and arrived at the conclusion that
> > the only way to deal with this is to use something that is extremely
> > unlikely to occur in actual file names in the first place. At the
> > same time, it should not contain any characters that have special
> > meaning for the shell, to avoid major inconvenience. ",," is the
> > simplest string I could come up with that seems to fulfill both
> > conditions.
> 
> I see now. However, a thought keeps coming to my mind: what if the
> user would like to create a file with ',,' in the name?

Well, tough shit I would say... :-)

Again, I picked something a user is extremely unlikely to want to
create, specifically to avoid this situation. We could try to come up
with some escaping scheme (e.g. ",,," to mean ",,"), but I'm not sure
it's really useful.

> It seems to me, however, that there will still be a need for
> shorthands for the names of particular translators (for example, 'gu'
> for gunzip) in the situations when these particular translators will
> have to be removed recursively from some files in a directory.

I'm not sure a shorthand is really necessary here: If someone has such
specific needs, he can just as well use the full name, i.e.
"foo,,-gunzip/".

As for the implementation, we would have a translator named "-gunzip",
which would again proxy the underlying directory tree, but ignoring any
gunzip translators present.

Of course, we do not want to implement the "-" variant of each
translator explicitely. Rather, we can just make them all links to a
generic filter translator, which always skips the translators matching
the invocation name (without the "-" of course), i.e. if invoked as
-gunzip it would skip any gunzip translators, as -mboxfs any mboxfs
translators etc.

Or perhaps implement them as simplistic shell scripts, that just do
"exec filter gunzip" etc., thus avoiding the invocation name magic.
(Which is frowned upon in some contexts, though I don't know the exact
reasons...)

Also, many of the shorthand translators can in fact simply be links to
the normal forms, e.g. "x" for xmlfs. Only the magic ones like "u" need
a more explicit implementation.

That's the beauty of this approach: We only use standard mechanisms, and
thus can implement the individual shorthands whatever way we want :-)

-antrik-