On Tue, Jun 3, 2008 at 5:10 AM, <
olafBuddenhagen@gmx.net> wrote:
"-u" is more tricky that "-gunzip", as it doesn't skip one specific
translator, but rather filters a whole class. While it is certainly
possible to implement this also by means of a generic filter translator
with some more complex rules, it might be easier to use a hardcoded
implementation, making "-u" a real translator on its own...
I think I'll try to implement a translator '-u', because it seems to me that it
may take less time. Later I'd like to challenge the idea of a generic filter
translator, but not before I have finished the basic framework.
> Does it mean that, when the node 'file,,-u' is accessed, the proxy
> filesystem should generate a virtual node and stack upon it all
> translators which precede the 'u' translator?
Well, that's more or less the effect from the client's point of view.
Technically, it's a bit different: It should actually create a virtual
node that mirrors the original "file" node, and attach the "-u"
translator to this proxy node. "-u" will in turn present the client with
another virtual node, which corresponds to the original "file",
translated by all static translators present on the original node,
except for those that need to be skipped according to the filter rules.
Oh, I have finally got the general idea! No more mess in my head :-)
> By the way, could you please suggest an idea where I can find out how
> to list the entries of a stack of translators?
Check the implementation of open() in glibc. This function normally
follows all translators, or none (when using O_NOTRANS); but it should
give you an idea how translator traversal works, and allow you to come
up with a more selective implementation...
Thank you! I am still not very deep into the code of glibc, but now I know
where to dig further.
"showtrans" only lists passive translators. "fsysopts" is used to query
active ones. Checking its implementation should also be helpful.
I'll check the code of this programs.
> Actually, I should start with the question: should the proxy create
> its own stacks of translators or just stack active translators?..
The dynamic translators specified through the special file name suffixes
should be attached (as active translators) to the proxy node.
The proxy node itself, as explained in the provious mail, probably
sometimes needs to mirror the original node ignoring any (static)
translators attached to it, so things like "-gunzip" can explicitely
check the static translators, and follow them selectively; while in
other cases (like "gunzip"), it needs to mirror the node already
translated by any static translators present. ("gunzip" knows nothing
about the proxying, and thus unlike the "-gunzip" won't follow the
static translators explicitely.)
For now, it appears to me that the most generic implementation should
consider creating two mirror nodes for every node in the underlying
filesystem; one should mirror the bare underlying node, with no
translators at all; the other should mirror the same node with the whole
collection of static translators stacked upon it. Normal translators will
be stacked on the second mirror node, while special translators will
use the first mirror node to filter the translator stack, for example.
It seems to me that no actual classification of translators into normal
and special should be done, I mean that all translators should be
allowed to access both mirror nodes. Is it so, or do I overlook
something?
> When you say scripts, do you mean bash scripts or other scripting
> languages?
I was actually thinking of shell scripts here. Other languages could be
used as well, but probably would be overkill for a simple launcher :-)
Of course, instead of a simple launcher, you could implement some more
complicated scripts, if it seems useful...
As far as I can see, scripting languages are very popular nowadays. I do
agree that shell scripts would be most appropriate for a simple launcher,
but later it might prove useful to integrate some other scripting languages
into the namespaced translator selection project, at least for the sake of
attracting people :-) (Of course, I'm not even thinking of that before
completing the basic framework)
> What really leaves me confused is how the interaction between the
> proxy filesystem and the script should take place...
[...]
Or is your question more general: How the interaction between the main
namespace proxy and the special translators like "-gunzip" or "-u"
works?
Oh yes, indeed, my question is more general :-) At the moment I don't have
a clear idea of how to set up a proper communication channel between the
main proxy and the special translators. Something behind my consciousness
whispers: 'FIFO', but I've never worked with FIFO's before. However, I
suppose that such questions appear in my mind because I've not yet read
the appropriate code. Therefore, I'll abandon this problem for a while and
will get back to it after my understanding of libnetfs has become absolutely
clear.
> > And if not, when we stack a '-gunzip' translator upon a 'gunzip'
> > > translator, don't we get into redundancy somehow?
> >
> > I don't see any redundancy...
Yes, indeed, now I don't see any redundancy either. My problem was that
I didn't understand some fundamental things.
Now I'd like to ask such a question: suppose there is a node 'file' with static
translators 'x', 'u', 'y', and 'z' set upon it. When I ask the proxy to fetch me
'file,,-u', the proxy loads the '-u' translator, which takes the virtual node
mirroring the node 'file' with no translators, scans the stack of translators
up to 'u', creates a new virtual node mirroring 'file' and stacks 'x' upon it.
In the end the user gets 'file,,x'. Right?
What if I try to set another '-u' on the resulting node? Should the proxy
ignore it somehow?
What if I try to set another special translator on the resulting node? The
special translator would probably like to perform the procedure described
above, but there is only one virtual node 'file,,x' created by '-u' and no
auxiliary mirrors. How shall the main proxy manage such a situation?
> I didn't suppose that translators could be _stacked_ or _executed_. As
> far as I can get, executing a translator implies that it is not
> stacked anywhere, doesn't it?
No idea what you mean :-(
My problem was that I did not understand the fact that there should be two
conceptual groups of translators (normal and special) which would operate
on different mirror nodes.
> I was thinking about the stack containing a '-gunzip' upon 'gunzip'...
Yeah, I understood that. And I don't see how this is redundant.
"gunzip" is a static translator, which would normally present the
underlying node in gunzipped form. "-gunzip" is a special dynamic
translator that skips the static translator, thus presenting the
untranslated contents of the node. Nothing redundant about that...
Of course, now I get the idea.
> > The example shows a real problem though: For "-gunzip" applied this
> > way to take any effect, the "-gunzip" translator must be executed
> > *before* opening the underlying node -- before any static
> > translators that might be present on the node get traversed. When
> > applying a "normal" translator like gunzip on the other hand, it
> > should be simply be stacked on top of anything that might already be
> > present, just like stacking static translators. Not sure about the
> > best behaviour when applying a magic translator like "u" or "-u"
> > recursively -- in this case, I'm not sure it's necessary to apply it
> > before opening the node; but on the other hand it shouldn't hurt to
> > do so...
>
>
> Am I right when I infer from these words of yours that there is no way
> to ignore a translator which has been traversed already?
Good question actually...
Every translator knows the node it is attached to. I don't know though
whether there is any standard RPC that would allow to query this
information. If it turns out there is, this would actually simplify
things a lot -- "-gunzip" could just ask for the original (untranslated)
node under the static "gunzip" translator; no need for special handling
to make sure "-gunzip" gets active before following static translators
:-)
It somehow appears to me at the moment that the usage of two virtual
nodes mirroring the real node with and without translators might be
a guaranteed solution, even if such a standard RPC does not exist.
In this case '-gunzip' will just choose the untranslated mirror, won't it?
> If the proxy filesystem will create its own translator stacks (as I
> have also supposed above), static translator stacks will remain
> unchanged, right?
Yes. The static translators are never touched. They might be ignored
when presenting the virtual nodes, but that doesn't affect their
presence on the real, underlying filesystem.
That's absolutely clear now.
> And here I completely fail to imagine the mechanism by which the proxy
> will allow ignoring static translators...
That's trivial. To skip *all* translators present on a node, just use
O_NOTRANS. Skipping selectively requires a bit more work, as there is no
standard glibc function allowing for that (AFAIK); but I'm pretty sure
it can be done using the same underlying mechanisms.
Yes, indeed, the same idea came to my mind when I was reading
the previous paragraphs. Of course, I will look into the code of
'open' to implement selective skipping.
> I'm really sorry if my questions look silly, [...]
They don't. In fact, discussing those questions has already greatly
improved my own understanding of these matters :-)
That's great! :-) I hope my questions will never become redunant, though :-)