On Mon, Dec 8, 2008 at 9:28 AM,
<olafBuddenhagen@gmx.net> wrote:
On Thu, Dec 04, 2008 at 06:49:12PM +0200, Sergiu Ivanov wrote:
> On Wed, Dec 3, 2008 at 3:40 AM, <
olafBuddenhagen@gmx.net> wrote:
> By proxying control ports do you mean creating shadow nodes containing
> ports to the real control ports?
Well, the terminology is totally wrong: We are not creating any *nodes*
here. Nodes are file/directory objects provided by a filesystem server,
and accessed through a port of type file_t, which can be obtained with
dir_lookup (or fsys_getroot). io_t is the same as file_t AIUI.
Control ports are of type fsys_t, and are returned by
file_get_translator_ctrl or file_getcontrol. The corresponding object is
the filesystem itself.
So strictly speaking I'm talking of proxying *filesystems* here. We
create a virtual filesystem object, return the control port to the user,
and forward user requests to the corresponding real filesystem control
port.
That's clear. I'll put my question differently: are we going to return
the control ports (that is, ports to filesystem) inside a instances of
struct node (provided libnetfs) as it happens in the case of magic
lookups?
Note that I often use the terms "node" and "file port" interchangeably,
if the meaning is clear... Hope that doesn't cause any confusion.
I'll keep in mind your remarks about terminology and will raise my
hand immediately as I notice something unclear to me :-)
> If I understand your idea correctly, than the problem that bothers me
> is that each step in traversing the translator stack will require
> additional switch to nsmux.
Traversing the stack is done only once when the filter is started, so it
shouldn't be even noticable -- a few additional context switches are
negligible compared to the process startup... Especially as in practice
translator stacks will usually be pretty shallow: depth 1 being by far
the most common case.
Ah, really, I see. I somehow failed to see that the filter traverses
the underlying translator stack just once... Also I used to think
there were normally going to be more translators in the stack,
although I really cannot see other equally frequent cases as a
one-translator stack.
Checking to make sure I understand it correctly now: If a client invokes
dir_lookup, and the server finds some other translator in the path of
the lookup, it will get the translator's root node, and return that to
the client, along with the remaining file name components, and with
RETRY_REAUTH, right?
Hm, things are rather crooked here, and I must acknowledge that I did
not realize this. What I see in the if block starting at line 186 in
dir-lookup.c of libnetfs is that usually retries are not forwarded to
the client when translators are encountered (I wonder whether it was
this what you meant in the first strategy of doing lookup). However,
if the translator is symlink (line 289) and the link target is
specified as absolute path, a magic retry is requested (line 322).
So, as a conclusion, I must say that the standard implementation (and
hence my own) *does* some forwarding of retries, but it still handles
some of the retries on its own...
Sorry for confusion...
And what happens when doing a lookup for ".." on the root of a
filesystem? I would guess that it returns the filesystem's underlying
node, with ".." as the retry name, and RETRY_REAUTH. Is this about
right?
I'm not really sure, but as I can see from the libnetfs'
implementation of netfs_S_dir_lookup (dir-lookup.c, starting at line
112) it's not really so. As far as I can guess from the names of
variables and the scarce comments in the declaration of struct peropen
in /usr/include/hurd/netfs.h, just the parent node of the translator's
root node is returned with FS_RETRY_REAUTH.
BTW, I realized another rather serious problem: I did suggest that nsmux
doesn't need to proxy file nodes, thus saving a lot of overhead. This
was based on the assumption that from a file node no further lookups are
possible, and thus no need to handle magic. Unfortunately, this
assumption was wrong: It's always possible to invoke file_getcontrol and
then fsys_getroot, and do lookups from there...
Well, do you suggest proxying file nodes, too? Shall change my code to
proxy all nodes? I wonder, whether it is very bad that further lookups
are possible... Actually, right now I fail to see any reasons, why
this is so very bad. Could you please point some out?
What I am also thinking of is the fact that if we proxy file nodes,
too, many more instances of libnetfs struct node will be created,
which will slow down commands like ls -R and will, probably, have
other negative consequences.
> > The first variant is simpler and more efficient, and I think it
> > should work well in standard situations. (libc abstracts the retries
> > anyways.) I wonder though whether in some special cases it might
> > cause anomalies...
>
> You know, right now the nsmux does forward the retries
Interesting -- I assumed the opposite :-)
I'm a little surprised, as in another similar situation you actually
advocated hiding the retry from the client... (After starting a dynamic
translator, and continuing the lookup.)
Yes, I did... That was because I did not understand how lookup
works...
However, in view of the information I grasped while peering into
libnetfs' dir-lookup.c, I won't really be so sure that I know anything
about it now...
> BTW, could you give an example of an anomaly with the first approach?
> I fail to strain my imagination sufficiently to see any...
I can't give any specific example either. But I believe there might be
situations where it's necessary for a client to see filesystem
boundaries, instead of just transparently retrying. In fact, the filter
is one use case where it is necessary to deal with translators
explicitely... (Here we don't have issues with retries, because we don't
do a real lookup; but I suspect that in other situations it might be
different.)
I have a premonition that this issue might in fact come up as soon as we
start thinking seriously about the recursive magic we postponed for now.
Maybe, it's time we started discussing about this problem, too? The
question how the filter should work proved to be tightly connected
with recursive magic... Shall I start a new thread, what do you say?
> > I think I see now where the confusion comes from: Every dynamic
> > translator gets its *own* shadow node to sit on. When dynamic
> > translators are stacked, each new one gets a shadow node mirroring
> > the underlying translator. Each dynamic translator is only seen by
> > its own clients, but never by an underlying dynamic translator, just
> > as it is never seen by an underlying static translator...
>
> Yes, this is the main problem about traversing both static and dynamic
> translator stacks.
Why is this a problem? I fear that there is still some deep
misunderstanding lingering here...
Right now, when nsmux does not proxy control ports, once the filter is
provided with the control port to the very first translator in the
*static* translator stack, nsmux cannot know what the filter is
doing. This means that when the filter finishes travelling across the
static translator stack, it has no possibility of traversing the
*dynamic* translator stack it is member of.
I'd rather suppose that this is not really a misunderstanding, but a
consequence of thinking (or at least trying to think) in the same way
as nsmux thinks *now* :-)
> > Setting a dynamic translator on top of another dynamic translator is
> > not really different from setting it on a static one.
>
> You mean that setting a dynamic translator on a dynamic translator is
> similar to setting a static translator on a static translator, do I
> get it right?
No, I mean that setting a dynamic translator on a dynamic translator is
similar to setting a dynamic translator on a static translator.
There is really no difference whether the underlying node is provided by
a dynamic translator, a static translator, or just a "normal" directory
entry. The procedure for setting a dynamic translator is always the
same. (Create a shadow node mirroring the underlying node; set the
dynamic translator on that; continue lookup.)
Hm... Suppose nsmux is asked to fetch 'file,,x,y'. Right now it
creates a shadow node, sets translator 'x' on the shadow node and then
sets translator 'y' immediately on top of translator 'x'. Do you mean
that nsmux should create a new proxy node for translator 'y'? That is,
shall it work in the following way:
1. Create a shadow node mirroring file 'file';
2. Set translator 'x' on this shadow node;
3. Create a shadow node mirroring the normal port of translator 'x';
4. Set translator 'y' on this *second* shadow node?
> [...] when I tried
> reopening ports to files using file_name_lookup* and dir_lookup using
> different flags, I only got a copy of the same port; there seemed to
> have been no actual reopening.
[...]
But as for your observation, this really can't be true -- it is always
possible to open the same file multiple times, possibly with different
mode.
(I actually opend the same file multiple times as a test case when
investigating port exhaustion issues...)
I probably have explained myself badly. Yes, of course, there is no
problem reopening the same multiple times with different modes. The
problem I was speaking about sounds as follows. Suppose you have a
port to a file, but you do not know the file name. You cannot reopen
the same port with different flags.
Aside from repeating the whole lookup, it's also possible AIUI to reopen
a node (possibly with different mode) by looking up "" on it -- so in a
slightly altered variant, the above idea *should* indeed work.