Re: What shall the filter do to bottommost translators

[olafBuddenhagen]

Hi,

On Thu, Dec 04, 2008 at 06:49:12PM +0200, Sergiu Ivanov wrote:
> On Wed, Dec 3, 2008 at 3:40 AM, <olafBuddenhagen@gmx.net> wrote:

> By proxying control ports do you mean creating shadow nodes containing
> ports to the real control ports?

Well, the terminology is totally wrong: We are not creating any *nodes*
here. Nodes are file/directory objects provided by a filesystem server,
and accessed through a port of type file_t, which can be obtained with
dir_lookup (or fsys_getroot). io_t is the same as file_t AIUI.

Control ports are of type fsys_t, and are returned by
file_get_translator_ctrl or file_getcontrol. The corresponding object is
the filesystem itself.

So strictly speaking I'm talking of proxying *filesystems* here. We
create a virtual filesystem object, return the control port to the user,
and forward user requests to the corresponding real filesystem control
port.

Note that I often use the terms "node" and "file port" interchangeably,
if the meaning is clear... Hope that doesn't cause any confusion.

> If I understand your idea correctly, than the problem that bothers me
> is that each step in traversing the translator stack will require
> additional switch to nsmux.

Traversing the stack is done only once when the filter is started, so it
shouldn't be even noticable -- a few additional context switches are
negligible compared to the process startup... Especially as in practice
translator stacks will usually be pretty shallow: depth 1 being by far
the most common case.

> > I'm not sure what to do for normal lookups, though. There are two
> > possibilities: Either, when a user does a lookup on the virtual tree
> > exported by nsmux, nsmux could do the lookup on the real tree it
> > mirrors in one go, handling any necessary retries internally, and
> > returning only (a proxy of) the requested final node to the user.
> >
> > The other possibility would be to actually forward any retries, so
> > the user has to handle them, just as if accessing the real tree
> > directly. In the latter case we would have to return control ports
> > when forwarding retries AIUI, which we would have to proxy too.

I have finally looked at the lookup/retry code, and I think I mostly
grasp the machanism now. (I don't have a complete understanding though,
as I only checked the client-side code, not the server implementations.)

I see now that the above statement was totally wrong.

Checking to make sure I understand it correctly now: If a client invokes
dir_lookup, and the server finds some other translator in the path of
the lookup, it will get the translator's root node, and return that to
the client, along with the remaining file name components, and with
RETRY_REAUTH, right?

And what happens when doing a lookup for ".." on the root of a
filesystem? I would guess that it returns the filesystem's underlying
node, with ".." as the retry name, and RETRY_REAUTH. Is this about
right?

BTW, I realized another rather serious problem: I did suggest that nsmux
doesn't need to proxy file nodes, thus saving a lot of overhead. This
was based on the assumption that from a file node no further lookups are
possible, and thus no need to handle magic. Unfortunately, this
assumption was wrong: It's always possible to invoke file_getcontrol and
then fsys_getroot, and do lookups from there...

> > The first variant is simpler and more efficient, and I think it
> > should work well in standard situations. (libc abstracts the retries
> > anyways.) I wonder though whether in some special cases it might
> > cause anomalies...
> 
> You know, right now the nsmux does forward the retries

Interesting -- I assumed the opposite :-)

I'm a little surprised, as in another similar situation you actually
advocated hiding the retry from the client... (After starting a dynamic
translator, and continuing the lookup.)

> BTW, could you give an example of an anomaly with the first approach?
> I fail to strain my imagination sufficiently to see any...

I can't give any specific example either. But I believe there might be
situations where it's necessary for a client to see filesystem
boundaries, instead of just transparently retrying. In fact, the filter
is one use case where it is necessary to deal with translators
explicitely... (Here we don't have issues with retries, because we don't
do a real lookup; but I suspect that in other situations it might be
different.)

I have a premonition that this issue might in fact come up as soon as we
start thinking seriously about the recursive magic we postponed for now.

> > I think I see now where the confusion comes from: Every dynamic
> > translator gets its *own* shadow node to sit on. When dynamic
> > translators are stacked, each new one gets a shadow node mirroring
> > the underlying translator. Each dynamic translator is only seen by
> > its own clients, but never by an underlying dynamic translator, just
> > as it is never seen by an underlying static translator...
> 
> Yes, this is the main problem about traversing both static and dynamic
> translator stacks.

Why is this a problem? I fear that there is still some deep
misunderstanding lingering here...

> > Setting a dynamic translator on top of another dynamic translator is
> > not really different from setting it on a static one.
> 
> You mean that setting a dynamic translator on a dynamic translator is
> similar to setting a static translator on a static translator, do I
> get it right?

No, I mean that setting a dynamic translator on a dynamic translator is
similar to setting a dynamic translator on a static translator.

There is really no difference whether the underlying node is provided by
a dynamic translator, a static translator, or just a "normal" directory
entry. The procedure for setting a dynamic translator is always the
same. (Create a shadow node mirroring the underlying node; set the
dynamic translator on that; continue lookup.)

> > If we find it desirable after all to open the mirrored node
> > immediately for some reason, there is yet another possible variation
> > I think: The shadow node could get only an unauthenticated port to
> > the mirrored node on creation, and do the proper open only once it
> > gets an actual client request.
> 
> I'm not sure it is possible to make it like that, because when I tried
> reopening ports to files using file_name_lookup* and dir_lookup using
> different flags, I only got a copy of the same port; there seemed to
> have been no actual reopening.

Well, this idea was also based on my former misunderstanding of the
lookup mechanism...

But as for your observation, this really can't be true -- it is always
possible to open the same file multiple times, possibly with different
mode.

(I actually opend the same file multiple times as a test case when
investigating port exhaustion issues...)

Aside from repeating the whole lookup, it's also possible AIUI to reopen
a node (possibly with different mode) by looking up "" on it -- so in a
slightly altered variant, the above idea *should* indeed work.

But there is not really much point in considering this further, unless
we actually find a good reason to open a port as soon as the shadow node
is created...

-antrik-