Re: Namespace-based translator selection; project details

[olafBuddenhagen]

Hi,

On Tue, Jun 03, 2008 at 07:50:14PM +0300, Sergiu Ivanov wrote:
> On Tue, Jun 3, 2008 at 5:10 AM, <olafBuddenhagen@gmx.net> wrote:

> > "-u" is more tricky that "-gunzip", as it doesn't skip one specific
> > translator, but rather filters a whole class. While it is certainly
> > possible to implement this also by means of a generic filter
> > translator with some more complex rules, it might be easier to use a
> > hardcoded implementation, making "-u" a real translator on its
> > own...
> 
> 
> I think I'll try to implement a translator '-u', because it seems to
> me that it may take less time. Later I'd like to challenge the idea of
> a generic filter translator, but not before I have finished the basic
> framework.

Sounds reasonable.

> > The proxy node itself, as explained in the provious mail, probably
> > sometimes needs to mirror the original node ignoring any (static)
> > translators attached to it, so things like "-gunzip" can explicitely
> > check the static translators, and follow them selectively; while in
> > other cases (like "gunzip"), it needs to mirror the node already
> > translated by any static translators present. ("gunzip" knows
> > nothing about the proxying, and thus unlike the "-gunzip" won't
> > follow the static translators explicitely.)
> 
> For now, it appears to me that the most generic implementation should
> consider creating two mirror nodes for every node in the underlying
> filesystem; one should mirror the bare underlying node, with no
> translators at all; the other should mirror the same node with the
> whole collection of static translators stacked upon it. Normal
> translators will be stacked on the second mirror node, while special
> translators will use the first mirror node to filter the translator
> stack, for example.

Note that the virtual nodes are always created on demand, the moment
they are accessed; so you would never actually create both variants up
front. But conceptually, you indeed could think of two virtual nodes
used alternatively...

> It seems to me that no actual classification of translators into
> normal and special should be done, I mean that all translators should
> be allowed to access both mirror nodes. Is it so, or do I overlook
> something?

The bit you are overlooking is that a translator is attached to a single
node the moment it is started. It doesn't decide itself where it wants
to be attached...

Nevertheless, there is a lot of merit in your observation: I realise now
that there is really no reason to externally classify the translators,
and attach them them to a different node up front. Instead, all
translators could just be attached to the node with all static
translators present; and the proxy would provide an additional interface
an all the virtual nodes, explicitely allowing to ask for the same node,
but untranslated.

Normal translators like "gunzip" would just use the underlying node
directly, forming an ordinary translator stack. Filter translators on
the other hand would use the additional interface to got the
untranslated node corresponding to the translated one they are attached
to, and work on that.

> As far as I can see, scripting languages are very popular nowadays. I
> do agree that shell scripts would be most appropriate for a simple
> launcher, but later it might prove useful to integrate some other
> scripting languages into the namespaced translator selection project,
> at least for the sake of attracting people :-) (Of course, I'm not
> even thinking of that before completing the basic framework)

Well, this is really orthogonal. Of course one could use Flavio's Lisp
bindings to write "-u" for example -- just like for writing any other
translator...

The whole point of this idea is that all the special syntax, like "-u",
is provided by ordinary translators, and we can use any method we like
for the implementation of these :-)

> At the moment I don't have a clear idea of how to set up a proper
> communication channel between the main proxy and the special
> translators.

The communication channel is established the moment a translator is
launched: The translator is attached to the underlying node upon
startup. Part of this process involves the translator getting a port to
the node on the parent FS it is attached to. (The virtual node on the
main namespace proxy, in our case.)

Most of the requests on this port are simply forwarded by the proxy to
the underlying "real" nodes, so it behaves just like the translator had
been indeed attached directly to the "real" nodes. Some requests however
are more explicitely handled by the proxy -- like for instance providing
an untranslated (virtual) node corresponding to the translated one...

> suppose there is a node 'file' with static translators 'x', 'u', 'y',
> and 'z' set upon it. When I ask the proxy to fetch me 'file,,-u', the
> proxy loads the '-u' translator, which takes the virtual node
> mirroring the node 'file' with no translators, scans the stack of
> translators up to 'u', creates a new virtual node mirroring 'file' and
> stacks 'x' upon it. In the end the user gets 'file,,x'. Right?

Exactly. You got it :-)

I wouldn't write it as "file,,x", though -- this is too confusing. The
result of actually opening "file,,x" would be quite different: It would
stack a new dynamic "x" translator on top of the "file" node with all
it's static traslators present -- "x", "u", "y", "z", and now another
"x"...

> What if I try to set another '-u' on the resulting node? Should the
> proxy ignore it somehow?

No need to. It will have no further effect, but should do no harm.

> What if I try to set another special translator on the resulting node?
> The special translator would probably like to perform the procedure
> described above, but there is only one virtual node 'file,,x' created
> by '-u' and no auxiliary mirrors. How shall the main proxy manage such
> a situation?

Now it's getting really confusing... Maybe we should start drawing
pictures for illustration :-)

I have no idea though what you mean by "auxillary mirrors". I don't see
any problem with this situation; no need for any special handling...

Note that the virtual node mirroring the original "file" node with the
static "x" translator applied (what you call "file,,x", though as I
explained above this is confusing), is actually created by the main
proxy, upon request from "-u". "-u" first opens a mirror of the
untranslated node, and then follows only the "x" translator, which
causes the main proxy to create another mirror, this time with the "x"
applied. "-u" then returns that as the result to be passed to the
client.

When another filtering translator is applied, say "-foo", this one will
again ask for a mirror of the untranslated node, and check what
translators exist on it. It gets only "x" this time, as "x" is the only
translator present on the virtual node previously created. If the filter
rule of "-foo" says that "x" should be filtered, foo will return a
mirror of "file" without any translators; otherwise, it will just again
return the mirror of "file" with "x" applied -- the very same (virtual)
node it is attached to. In other words, "-foo" won't change anything if
the filter rule doesn't require skipping "x"; it just passes through the
previous node.

> > Every translator knows the node it is attached to. I don't know
> > though whether there is any standard RPC that would allow to query
> > this information. If it turns out there is, this would actually
> > simplify things a lot -- "-gunzip" could just ask for the original
> > (untranslated) node under the static "gunzip" translator; no need
> > for special handling to make sure "-gunzip" gets active before
> > following static translators :-)
> >
> 
> It somehow appears to me at the moment that the usage of two virtual
> nodes mirroring the real node with and without translators might be a
> guaranteed solution, even if such a standard RPC does not exist. In
> this case '-gunzip' will just choose the untranslated mirror, won't
> it?

Well, yes. As explained above, "-gunzip" could always ask for the
untranslated node by means of a special interface provided by the main
proxy. But if there is some standard way to ask any translator for the
underlying node, we wouldn't need the special interface. It would
simplify things.

-antrik-