Re: Unionmount. Basic details

[Carl Fredrik Hammar]

Hello,

On Sun, Apr 26, 2009 at 06:28:51AM +0200, olafBuddenhagen@gmx.net wrote:
> On Thu, Apr 09, 2009 at 10:09:04PM +0200, Carl Fredrik Hammar wrote:
> > On Wed, Apr 08, 2009 at 07:10:26PM +0200, olafBuddenhagen@gmx.net
> > wrote:
> 
> > > while unionfs must be able to handle an arbitrary number of merged
> > > locations, for unionmount it's always exactly two. This simplifies
> > > the implementation and policy decisions, and might even call for
> > > different optimization approaches -- so it seems quite possible that
> > > the code base will diverge considerably after the fork...
> > 
> > Why stop at two?  Mounting several file systems at once seems
> > perfectly reasonable to me.
> 
> I do not see much point in mounting several file systems at the same
> time. Just mount them one after the other. This is less efficient of
> course (as you get a whole stack of unionmount translators), but
> probably not very common anyways...

It's easier to manipulate a single translator than a stack.  Consider the
case of unioning three file systems.  To remove the middle one you'd have
to unmount the two top ones and then remount the top.  With a single
translator you can get away with using fsysopts, which could possibly
be atomic.

I'm not so sure it's uncommon case.  Though I guess time will tell.

> [snip]
> 
> > My first thoughts of anonymous file systems got me thinking of an
> > ``anonfs'' translator that launched translators encoded in the paths.
> > But the thought of encoding paths in a path gave me shudders.
> 
> Isn't that more or less what nsmux does?...

Almost, I was thinking of more of a /anonfs directory to create anonymous
file systems, e.g. `/anonfs/unionfs foo bar/baz'.  But would get ugly
if foo or bar contained `/'.

> > Then it got me thinking of a shell utility, e.g.:
> > 
> >    letfs foo "/hurd/nsmux foo" -- ... -- \ settrans /hurd/unionfs foo
> >    %1 %2 ... %n
> > 
> > Where %i is replaced with /dev/fd/${fd to %i's root}.
> 
> I don't really get the idea...

It would launch a number of translators, passing file descriptors to their
roots to a command where variables are expanded to paths specifying these
file descriptors using glibc's /dev/fd functionality.  In my example,
translators are separated by `--', each having settrans like syntax
(not sure why I put quotes in there), and after the last `--' comes the
command with replacement variables.

It's reminiscent of bash's process substitution where you can pass a
pipe as an argument, e.g. ``ls|head'' becomes ``head <(ls)''.  Indeed,
it's where I got the idea in the first place.

> Anyways, what I want is basically a way to launch a translator as part
> of another command which is using it -- a bit like traditional UNIX
> pipes. I think nsmux can cover many of the use cases; though it probably
> would get rather icky for the more complicated ones... Some more generic
> anonymous translator mechanism could be useful -- but I have no clue how
> it would work exactly.

I think letfs would do exactly what you want.  :-)

> > This leaves the question of how and when to make the anonymous file
> > systems go away.
> 
> This is actually quite easy for anonymous translators: They can go away
> as soon as all clients have closed the connection (and any outstanding
> processing is finished) -- as it is anonymous, nobody other than the
> initial client should be able to connect to it, so we know it won't be
> needed afterwards. In this regard too it is very similar to programs
> connected with anonymous pipes...
> 
> Deciding when a translator should go away is much more tricky for normal
> static translators -- we don't know whether they will be still needed
> after all previous clients closed the connection.

Well, yes.  But all current translators assume that they are static
and thus doesn't go away when there are no connections (though some do
timeout eventually).  Extending all translators to also support this is
not a good option IMHO.

We could try to detect when it's no longer used.  But this seems tricky,
and requires that ports are proxied.  Worse, since we don't know which
interfaces a translator implements we'd have to intercept all messages
and proxy all incoming and outgoing ports.  And with such a heavy
handed approach, the translator might never die if some ports are used as
weak references and should not keep the translator from quiting.

Another, much more basic option might be to stop it at a predictable
event.  For letfs the natural event would be when the sub-commands exit.
This isn't perfect, e.g. the command might pass a port to another process.
This might seem like an extravagant use case, but this would happen in
my letfs example with settrans.

A third option might be to continually ask it to go away until it does.
But I'm no fan of polling like this, but might be acceptable in addition
to the previous option.

Regards,
  Fredrik