Re: What shall the filter do to bottommost translators

[olafBuddenhagen]

Hi,

On Mon, Dec 22, 2008 at 07:19:50PM +0200, Sergiu Ivanov wrote:
> On Thu, Dec 18, 2008 at 4:13 PM, <olafBuddenhagen@gmx.net> wrote:
> > On Sun, Dec 14, 2008 at 11:34:57PM +0200, Sergiu Ivanov wrote:

> > > I'll put my question differently: are we going to return the
> > > control ports (that is, ports to filesystem) inside a instances of
> > > struct node (provided libnetfs) as it happens in the case of magic
> > > lookups?
> >
> > Well, I don't know libnetfs... It *sounds* like it would be serious
> > abuse (if it's even possible at all) -- but perhaps the "node"
> > concept in netfs is actually more generic than it sounds...
> 
> I'm rather inclined to view the node concept introduced in libnetfs as
> a rather generic concept, and my question arose from such
> understanding. Still, I realize that such use will be abusive, so I'd
> like to ask now how would you suggest to implement this feature?

Not knowing libnetfs, I can only make rather wild guesses...

In a sense, we are implementing multiple filesystems: One for each
filesystem in the mirrored directory tree.

The most radical approach would be to actually start a new nsmux
instance for each filesystem in the mirrored tree. This might in fact be
easiest to implement, though I'm not sure about other consequences...
What do you think? Do you see how it could work? Do you consider it a
good idea?

But let's assume for now we stick with one nsmux instance for the whole
tree. With trivfs, it's possible for one translator to serve multiple
filesystems -- I guess netfs can do that too... Though it might be
tricky, because here we don't attach the translator to multiple
locations in the "outside" filesystem, but rather attach to one outside
location, while the other filesystems are in turn attached to nodes
served by nsmux itself -- I wonder whether this recursion would cause
any complications.

Even if it works, it might be too much overkill, though. The alternative
would be to override the default implementations of some of the fsys and
other RPCs dealing with control ports, so we would only serve one
filesystem from the library's point of view, but still be able to return
different control ports.

As we override the standard implementations, it would be up to us how we
handle things in this case. Easiest probably would be to store a control
port to the respective real filesystem in the port structure of every
proxy control port we return to clients.

Again, without digging into libnetfs myself, I really can't tell which
of these variants are preferable, or even possible...

> > The file_getcontrol()->fsys_getroot() sequence is a very odd one --
> > what would you provide as the "dotdot" node for fsys_getroot()?... I
> > don't think this possibility was really intended.

> Hm... I'm not sure I can understand properly what you are talking
> about. Why would this sequence be odd? I cannot think of any other
> possibilities to traverse a translator stack than using this sequence.

Eh? Traversing translator stacks -- just like normal lookup -- uses
file_get_translator_cntl(), not file_getcontrol(), right?

> As for the ``dotdot'' node, nsmux usually knows who is the parent of
> the current node; if we are talking about a client using nsmux, it is
> their responsibility to know who is the parent of the current node.
> 
> OTOH, I am not sure at all about the meaning of this argument,
> especially since it is normally provided in an unauthenticated
> version.

AIUI it is returned when doing a lookup for ".." on the node returned by
fsys_getroot(). In other words, it normally should be the directory in
which the translated node resides.

As the authentication is always specific to the client, there is no
point in the translator holding anything but an unauthenticated port for
"..".

My point was that if we obtain the control port with file_getcontrol(),
rather than file_get_translator_cntl() as in a normal lookup, we lack
the context: We can use file_getcontrol() to check what translator
serves a given file, but we don't know where the translator resides in
the directory tree. Thus, calling fsys_getroot() and doing further
lookups from there doesn't really seem to make much sense in this
situation... Which is why I think we might be able to get away just
ignoring this case.

> > Also, the fact that unionfs proxies all nodes, makes me somewhat
> > uneasy -- surely there must be a reason for that?...
> >
> Please correct me if I am wrong: proxying a node means the following:
> 
> 1. A client requests lookup. 2. nsmux looks up the required file. 3.
> nsmux creates a new libnetfs node containing a port to the file looked
> up 4. nsmux returns a port to the newly created node to the client

Yes.

> If I understand things correctly, then unionfs does *not* proxy all
> nodes. Similarly to nsmux, unionfs only proxies directory nodes.

Oh, I thought you said at some point that unionfs proxies all nodes...
Guess I mixed it up. If unionfs doesn't, I'm pretty confident we do not
need to do it either :-)

> > > Hm... Suppose nsmux is asked to fetch 'file,,x,y'.
> >
> > You mean "file,,x,,y" I assume?...
> >
> This syntax looks new to me :-) I don't remember seeing or using it
> before.

That's pretty strange... I indeed can't find me mentioning this syntax
anywhere in the list archives. But I'm almost sure I must have used it
on IRC at least...

Anyways, even if I never stated it explicitely, I don't see any reason
to consider using any *different* syntax. This is really the obvious
one. We use one suffix to set a dynamic translator, and then another
suffix to set a second one on top of it... No need to special-case this
in any way.

> > Wouldn't it be simpler and clearer to process only the first suffix,
> > and pass back any remaining ones as the retry_name, so that they
> > automatically get handled correctly when the client does the
> > retry?... (It seems to me that if you had done it that way, it would
> > have been clearer from the beginning how stacking of dynamic
> > translators is to be handled.)
> 
> Well, setting translators in a simple loop is a bit faster, since, for
> example, you don't have to consider the possibility of an escaped
> ``,,'' every time

Totally neglectible...

> (and this is not the only reason).

What are the others?

> Unfourtunately, I cannot really feel the advantage of using retries in
> the case of creating dynamic translator stacks.

Uniformity for one: You agreed that we should forward the retries when
crossing translator boundaries on the mirrored filesystem, and also when
continuing the lookup after starting a dynamic translator. Why should we
do differently when starting multiple dynamic translators, conflating
the lookup steps into one, instead of forwarding the retries after
starting each individual translator?...

> BTW, could you please expound on the question why would we need extra
> shadow nodes in a dynamic translator stack? I fail to see how this
> would make our life simpler :-)

It's not even about simplicity: It's about correctness.

It's harder to come up with a good example when dealing only with
single-node translators; but when considering translators serving
directories, it really becomes quite obvious: The first dynamic
translator exports a directory. Now in the original lookup we have
another magic suffix, so this directory is further translated. However,
further lookups can be performed on the directory, and such further
lookups might want to skip the second translator. They can actually use
another magic suffix to invoke a different second translator on top of
the same first one. Or someone might even set a static translator on the
directory provided by the first dynamic one...

In these cases it is crucial that we do *not* see the second dynamic
translator resulting from the original lookup, while accessing the
directory provided by the first dynamic translator. In other words, the
second dynamic translator must sit on a shadow node, just like the first
one. That's the definition of dynamic translators: They are visible only
for their own clients, but not from the underlying node. (No matter
whether this underlying node is served by another dynamic translator.)

Having said that, I still believe that this also simplyfies things, as
this way we don't need any special-casing when setting more than one
dynamic translator. We just start the first one, and push the ball back
to the client -- the resulting retry will make us start the second one
in exactly the same manner as the first.

-antrik-