Re: Unionmount. Basic details

[Carl Fredrik Hammar]

Hi,

On Wed, Apr 08, 2009 at 07:10:26PM +0200, olafBuddenhagen@gmx.net wrote:
> On Mon, Apr 06, 2009 at 06:58:23PM +0200, Carl Fredrik Hammar wrote:
> > On Mon, Apr 06, 2009 at 04:26:25PM +0300, Sergiu Ivanov wrote:
> 
> > So the only difference between unionmount and unionfs is the setup and
> > the shadow node, right?
> 
> Well, not quite. unionmount needs additional functionality for the
> internal node and related stuff; but on the other hand, the actual
> merging (unioning) requirements are simpler: while unionfs must be able
> to handle an arbitrary number of merged locations, for unionmount it's
> always exactly two. This simplifies the implementation and policy
> decisions, and might even call for different optimization approaches --
> so it seems quite possible that the code base will diverge considerably
> after the fork...

Why stop at two?  Mounting several file systems at once seems perfectly
reasonable to me.

> > If you're uncomfortable keeping around a process just to implement a
> > shadow node, consider implementing a dedicated shadow node server.
> > That just sitts on e.g. `/server/shadow' and passes out shadow nodes
> > in responce to RPCs of a new kind.
> 
> While there are some servers in the current Hurd that work like that, I
> don't think this is a good approach. We really want to move away from
> such centralization -- it limits scalability, lowers robustness, and
> increases complexity...

Yes, you are right.  I got a bit caught up in the moment, and didn't think
it through.

> If we really feel the need to run several instances of a translator in a
> single process (to reduce overhead), this should happen as transparently
> as possible, and probably be handled by a generic framework -- similar
> to translator stacking...

Seems like a potential use case for libmob.  We have discussed similar use
cases, but I must admit that I haven't considered a generalized ``hosting
service'' for mobile objects.

Not sure its a good idea, but I like it.  :-)

> > I think such a server might be a good idea in any case.  Shadow nodes
> > are already needed by nsmux (right?) and seem generally useful for
> > creating ``anonymous'' file systems.
> 
> As I pointed out in the other mail, the requirements here are really
> different from the shadow nodes in nsmux. Yet the observation about
> anonymous file systems is extremely to the point!

Yes, I'm a bit surprised it hasn't come up before.  The concept seems
obvious in hindsight.  I haven't been able to keep my mind of it since.
:-)

> I actually realized a couple of days ago that unionmount could probably
> be done by a combination of nsmux and unionfs: I think it should be
> possible to do something like
> 
>    settrans veth /hurd/unionfs veth veth,,eth-multiplexer
> 
> (I didn't want to bring this up yet, to avoid further confusion :-) )

Bootstrapping a unionmount of nsmux like might be tricky.  ;-)

> The funny thing is that while I was thinking about something like
> anonymous file systems multiple times in the past; and also had a vague
> realization that nsmux can serve most (if not all) use cases for that;
> and even was aware that this is what the example above uses -- somehow I
> never really consciously thought about the "anonymous" aspect as the
> common underlying idea up till now...

My first thoughts of anonymous file systems got me thinking of an
``anonfs'' translator that launched translators encoded in the paths.
But the thought of encoding paths in a path gave me shudders.

Then it got me thinking of a shell utility, e.g.:

   letfs foo "/hurd/nsmux foo" -- ... -- \
      settrans /hurd/unionfs foo %1 %2 ... %n

Where %i is replaced with /dev/fd/${fd to %i's root}.

This leaves the question of how and when to make the anonymous file
systems go away.  The only thing I can come up with is to do it when
the commands exits.  But this doesn't seem appropriate in general,
e.g. what if the command makes a server open an anonymous file system
and then exits.

However, for a unionmount it seems to make sense for them to go away
when unionfs does.  Just add a wait for unionfs to finish to make the
example above work this way.

Given that it seems natural for anonymous file system dependencies to go
away when the dependent one does.  Perhaps it is natural to add such a
``letfs'' feature to settrans itself (or a settrans variant).

> > The only real question is wheather they are independant enough to be
> > put in their own server.
> 
> Indeed, while we always try to think of the shadow nodes as separate
> translators, they are really quite intertwined with the rest of nsmux.
> But perhaps with anonymous translators as a more generic concept, the
> separation would be clearer...
> 
> However, it's probably a safer approach to duplicate the code in
> unionmount for now; and only once it works, and we know exactly what the
> requirements are, we should try to factor out the common
> functionality...

Agreed, this is usually for the best in such cases.

> PS. Sorry I didn't answer the mobility mail yet... Was too busy with
> GSoC preparations, and some silliness in the offline world. I hope I'll
> get around to it next week or so...

OK, I'm looking forward to it.  :-)

Regards,
  Fredrik