Re: Unionmount. Basic details

[Carl Fredrik Hammar]

Hello,

On Fri, Apr 10, 2009 at 08:35:07PM +0300, Sergiu Ivanov wrote:
> Carl Fredrik Hammar <hammy.lite@gmail.com> writes:
> > On Tue, Apr 07, 2009 at 12:20:40AM +0300, Sergiu Ivanov wrote:
> >> > [...]
> >> > If you're uncomfortable keeping around a process just to implement
> >> > a shadow node, consider implementing a dedicated shadow node server.
> >> > That just sitts on e.g. `/server/shadow' and passes out shadow nodes in
> >> > responce to RPCs of a new kind.
> >> > [...]
> >> 
> >> Yep, I'm really uncomfortable about keeping around processes, but in
> >> this case I'm inclined to think that spawning an extra process would be
> >> an overkill. I'm more inclined to the simpler variant of borrowing the
> >> code from unionfs, stripping off the unnecessary features and modifying
> >> the startup sequence to get a new translator.
> >
> > Well it isn't simpler in the sense that we'd need to maintain two very
> > similar yet different code bases.  Improvements to one would likely get
> > ported to the other.
> 
> I understand this, but I'd rather say that reusing the whole unionfs
> when we just need to join 2 (or maybe more, if antrik says it's all
> right ;-) ) together is an overkill: unionfs implements a couple of
> extended functions (not required by unionmount) and modifications in
> which are also likely to get in the way of normal operation of some
> basic functions (as it sometimes happens). Also, the operation of
> launching unionfs in the way you suggest is fairly tricky, it seems to
> me.
> 
> Anyway, I'm not sure whether bringing some details in the code in
> unionmount up to date will require ``porting'', since I'm going to touch
> only a very small portion of the code, leaving the bulk of it intact.
> 
> Also, I'm not aware of anybody still doing any changes to unionfs :-)

I don't know the current state of unionfs myself, but I'm assuming it
still has bugs.  And I'm not (yet) convinced that any rule you'd add to
unionmount would be not be useful it unionfs or the other way around.  If
unionmount uses unionfs it benefits from improvements to it automatically.

Also, in many ways unionfs seems like an good candidate to make use of
libmob which I'm working on.  Making that that change would hopefully
not be too extensive, but it would not be trivial.

> >> This would yield faster code (no extra context switch, the shadow
> >> node is within the main translator) and more control over the merging
> >> functionality.
> >
> > Isn't the shadow node used mostly by the mountee?  If so it will be used
> > through RPCs in any case.
> >
> > I don't think there would be any extra context switches, they would
> > just be divided between different processes.  An extra process per mount
> > is no big deal IMHO.
> 
> This is true that in the greater part of the operation of unionmount
> there will be no difference in speed (the difference will be at startup,
> of course). However, I still don't have sufficiently compelling reasons
> to considering making the startup sequence more sophisticated.

I don't think the start up sequence will be very complicated:

  1) open a port to the mountee's root
  2) wrap it in a file descriptor (make sure it will be inherited.)
  4) fork and exec
     "settrans $st_args $mount_point \
      /hurd/unionfs $unionfs_args /dev/fd/$fd $mount_point"
  5) close port and file descriptor
  6) stack the go_away interceptor over (the new) $mount_point

Of course, you'll be the one stuck with handling the details.  In the end
it might be a lot more complicated than I think it is.

> Also, as antrik pointed out in some other mail, the node in unionmount
> is not really a shadow node conceptually.

Well I've never known exactly what a shadow node is in nsmux, so I wouldn't
know the difference.  No need to explain it to me though, I don't think it
has much bearing on this discussion.

> [snip]
>
> >> I like the idea of a shadow-node server :-) However, I would rather keep
> >> shadow-nodes inside processes, because they are cheap as compared to
> >> RPCs and you have full control over them. Moreover, some translators
> >> (like unionfs and hence unionmount) would like to keep a list of nodes
> >> they own and drop nodes when they don't need them. This policy would
> >> require more effort if a shadow-node server is involved.
> >
> > Does unionmount (or nsmux) need to do anything except keep them around and
> > destroy them when not needed anymore?  It seems easy enough to destroy
> > a node through RPCs, just unlink it or something.  If not it should be
> > easy enough to create a shadow_node_destroy RPC.
> 
> I still cannot see why having a shadow-node server is better than
> creating the shadow nodes in-process. Note that creating a new server
> will involve creating a new interface, which I'm inclined to consider a
> little bit of overkill...

Agreed, I have already conceded that such extra servers are not such a
good idea after all.  So there should be a unionmount process involved,
the issue remaining whether to reuse unionfs or not.

> > Thinking about it, clean-up presents some problems with ditching a
> > unionmount process.
> >
> > The mountee becomes effectively orphaned when unionfs dies.  With no
> > way to reach it from the file system, the user would have look up its PID
> > to kill it.
> >
> > To handle fsys_goaway requires that there be a proxy for the ports that
> > receives it and forwards it to the mountee and, if that doesn't fail,
> > forwards it to unionfs.  You probably know, but I'm hoping there's only
> > one such port.  In a pinch, this could also be implemented by a server
> > shared by all unionmounts.
> >
> > Now the question is what to do if unionfs crashes or is killed.
> > If there where a unionmount process it could detect it and ask
> > the mountee to go away.  Doing without is harder.  The only solution
> > I can come up with is letting unionfs inherit a port to a server that
> > kills the mountee when there are no send rights to the port.  But that
> > seems like a really weird hack.
> 
> Note that this reasoning starts with the supposition that we adopt the
> strategy you suggested (about inheriting ports) :-) I'm somehow inclined
> to see this as an advantage to the approach I am promoting, since the
> problems you describe here will be mainly absent in that case :-)
>
> When unionmount is a fork off unionfs code base, it has *full* control
> over the mountee and can do whatever it wants to it. When unionmount is
> requested to quit, it can gracefully shut down the mountee, thus doing
> all the cleanup required.

You'd still have to handle fsys_goaway differently than unionfs.  The
extra work in reusing unionfs would be to forward all other RPCs to the
node and setting it up.

> Having the mountee killed after unionmount is (forcibly) killed may not
> always be the desired effect, you know. I would rather have unionmount
> die on its own, but this is just an inclination, not a founded
> opinion. Personally, when I kill -9 a program, I am very much prepared
> to go after it and to collect all the garbage.

Oh I agree.  The problem concerned with crashes and non-KILL signals to
unionfs, without a unionmount to clean up.  A unionmount process could
trap non-KILL signals and handle them gracefully.

> > Of course, implementing such servers would be a long term goal.  This is
> > just to convince you that it's possible to reuse unionfs without an
> > additional process per unionmount.  Admittedly, these solutions are
> > aren't very pretty, but then again I don't think an extra process per
> > unionmount is a problem at all.  ;-)
> 
> I see :-) Well, I should say that an additional process per mount is not
> a problem, indeed, but I don't really see so far why we need this extra
> process so much, if can go without it? :-)

Well I've already mentioned the advantages of reusing unionfs.  And I do
think we should reuse it if it doesn't require changes to unionfs itself
that aren't natural extensions of it.

However, all the ifs and buts are starting to add up.  Perhaps it would be
safer to start by forking unionfs, and then rewrite it to reuse unionfs.

There is one more route you may want to consider.  As I mentioned in
my replies to antrik, unionmount is basically anonymous file systems +
unionfs.  You could write a utility to handle anonymous file systems
instead. Even if it turns out that a specific unionmount with special
rules is needed, we'd still get a very useful utility out of the
process.

Regards,
  Fredrik