Re: Unionmount. Basic details

[Sergiu Ivanov]

Hello,

<olafBuddenhagen@gmx.net> writes:
> On Mon, Apr 06, 2009 at 04:26:25PM +0300, Sergiu Ivanov wrote:
>> OTOH, implementing unionmount as extensions to translator libraries
>> would mean faster operation and automatic inclusion of the
>> functionality in *every* existing translators, but modifying something
>> would require more effort.
>
> I don't quite understand. What do you mean by "automatic inclusion"?...
>
> With the proxy, *any* translator could be union-mounted; while the
> client-side approach would require some adjustments at least for some
> translators, perhaps all.

I wanted to say that any translator gets union-mount-aware (though may
not be making proper use of this fact) immediately after rebuilding it
against the modified versions of translator libraries; one does not need
an extra translator installed on the machine. But I think this is a
feeble advantage indeed, so this ``automatic inclusion'' ``idea'' can be
discarded.

>> unionmount is expected to merge the filesystem on which it sits with
>> the filesystem exposed by the translator it is asked to start in
>> unionmount mode (further referred to as ``the Translator'').
>
> Nah, I think there are various clearer ways to name it: e.g. "target
> translator", or perhaps "inferior" (like in a debugger), or "mountee"...
> :-)

OK :-) Do you mind if I choose the ``mountee'' variant? It's the
shortest and the clearest in my opinion :-)

>> When unionmount is starting, it has (of course) a port to the
>> underlying node, which means that it has full access to its underlying
>> filesystem. Now, it can create a shadow node, mirroring the underlying
>> node and then set the Translator on this shadow node. The purpose of
>> this is to keep the Translator away from the real underlying node,
>> giving it at the same time all the information it should require.
>
> I don't think we should call it "shadow node": although there are some
> similarities, it seems to me that it's not quite the same as the shadow
> nodes in nsmux -- it would be confusing.
>
> For now, I suggest calling it "internal node" or "hidden node". We can
> still change the name later when the exact role becomes clearer.

OK. This is clear.

> It is not fully clear right now -- I realized that there is another
> decision to make: should the unionmount translator be directly visible
> as the translator attached to the mount node; or should it serve as a
> proxy, forwarding all requests on the filesystem port to the target
> translator -- thus making itself more or less transparent, so it appears
> as if the target was attached to the mount node directly?
>
> I tend towards the latter.

I tend towards the latter, too. In my vision, unionmount should only do
the union mount job, letting the mountee come into full power as soon as
the union mount stuff has all been done.

>> One of the advantages of this approach is that it reuses the ideas
>> implemented in unionfs to the maximum:
> [...]
>> This advantage could make the stand-alone approach cheap to implement,
>> at least as a proof-of-concept thing.
>
> It seems to me that the unionmount-specific quirks could be indeed more
> complex, or at least of a similar order, as the actual merging
> functionality that forms the core of unionfs. But you are right of
> course that the merging code can be reused from unionfs :-)

That's great :-) I was afraid there were some issues (which I failed to
notice) about reusing unionfs.

>> The idea is to modify the default implementation of netfs_S_dir_lookup
>> in such a way that it will yield control to the user-defined
>> netfs_attempt_lookup only in the case the requested file name could
>> not be found under the underlying node. (Obviously,
>> netfs_attempt_lookup can have priority instead, thus allowing the
>> translator to obscure some of the underlying nodes).
>
> Actually, I was initially thinking that the target translator would take
> precedence by default; and only realized later that for the use cases I
> envisioned, we actually need the other behaviour :-)
>
> In any case, you are right that we should support both variants. (Plan9
> also does allow both, BTW.)

OK, understood.

>> This approach will require adding some user-modifiable flag to the
>> corresponding translator library that will allow to switch on and off
>> this functionality, because not all translators would be happy running
>> in unionmount mode.
>
> It should be the user's decision whether to try union-mounting or not.

I was trying to think of a specific translator which would publish such
a specific file system that it will not make sense to union-mount
it. However, I couldn't think of something real (even /proc may be
union-mounted for some purpose), so I agree that it should be the user's
decision whether to use union-mounting or not.

> One question is, how does the user request the unioning functionality? A
> possible way would be adding options to the actual translators -- but
> this would probably have to be handled explicitely by each translator,
> making it rather ugly.
>
> Adding an option to settrans seems a more logical approach. However, we
> will need a way to pass this information to the translator somehow -- it
> might require changes to the translator startup procedure...

Indeed. The problem is to decide which approach is less uglier: having
each translator handle an extra argument or to modify something in the
startup procedure... I'm rather against modifying the startup procedure,
because it's tricky and may turn out to be either a piece of cake or
another headache.

All in all, I'd classify this issue as an argument against the
implementation of unionmount functionality as a translator library
extension :-)

>> The problem of this approach is that we go away from modularity,
>> loading lib{net,disk}fs with functionality for which they were not
>> initially designed.
>
> I don't see that as a problem.

Well, my personal sympathy is against including the unionmount
functionality in libnetfs :-) I'd rather keep it separate.

>> I am aware of the performance issue about extra context switches, but
>> if the unionmount translator will not give off ports to its own nodes,
>> but ports to *external* nodes (underlying filesystem nodes or those
>> published by the Translator), it will not take part in the frequent
>> (and most time-critical) I/O operations and act as an initial source
>> of ports only. I think it is reasonable for unionmount not to create
>> proxy nodes (in nsmux terminology), because I cannot presently invent
>> a use case where it will need control over the ports it gave off to
>> the client.
>
> Well, the situation is exactly the same as for ordinary unionfs (and
> nsmux): It should be fine in normal use cases -- although it might be
> possible to construct scenarious where it would break...
>
> Also note that this problem would only be half solved by the library
> approach: while the nodes provided by the target translator would not
> need context switches (as the merging happens in the same process), the
> nodes of the underlying file system require the same kind of proxying as
> with an external unionmount translator (or unionfs).

I'm not sure I can understand correctly what you mean. Why should the
underlying nodes of the filesystem require proxying? Why couldn't we
just give off ports to real nodes instead of creating proxies?..

Regards,
scolobb