Re: Hiding nodes with unionmount

[Sergiu Ivanov]

Hello,

On Tue, Aug 04, 2009 at 12:19:28PM +0200, Arne Babenhauserheide wrote:
> Am Dienstag, 4. August 2009 07:20:05 schrieb Sergiu Ivanov:
> > > You need chroot/bin/bash, I think.
> >
> > I tried putting bash into the chroot environment, but it didn't help.
> > Also, I tried invoking chroot only with the new root directory (since
> > the man page promises it should run the default shell), but it didn't
> > work either.  I guess I'm doing something wrong, so I'll ask tomorrow
> > at the meeting :-)
> 
> Please keep me posted on what it requires! 
> 
> I would assume the new root needs a full environment (that's what Gentoo 
> needs: All programs, libs, etc. But in Gentoo I also have to bind /dev and 
> /proc into the chroot environment)

Yes, you are right; Fredrik confirmed your thought.  I must admit that
I rarely used chroot in every-day life :-)
 
> > > As soon as the "subhurd needs root" issue is solved, it should allow
> > > every normal user to boot his own system.
> >
> > AFAIK, Zheng has made some progress in this area.
> 
> Maybe you can ask tomorrow how far he got. 

AIUI, Zheng has a partially working implementation of rootless
subhurd.  antrik asked him to make his changes public, even though
they may not be the ones bringing around the complete functionality.

> > Frankly speaking, I didn't even know about VCS filesystems, to tell
> > nothing of their being ridiculed :-(
> 
> I learned about them, when I dived into Mercurial. It seems people wanted to 
> do them for a long time now, but I think they failed at the "when should we 
> commit" question. 
> 
> Luckily we can answer that for private users (corporations have far harder 
> backup needs): We commit when we feel like it, and not at every write. 
> 
> Also we now have far more efficient and flexible VCS systems than ever 
> before. 

Yes, this is true.  I'd say the VCS filesystems topic should be
revisited and thought over more attentively.  Committing at some
intervals, dynamically adjusted to the disk load (like: ``commit more
often when more operations are done'') could be a nice (and simple)
solution for desktop systems.
 
> > > The difference here would be that we could define fixed steps for
> > > committing, for example logout.
> > >
> > > It could even mean that the system isn't saved till logout. All changes
> > > could be in memory, until we logout and commit.
> >
> > Sounds reasonable for desktop use in regions where power shortages are
> > rare.
> 
> The sole problem would be saving a big file could require quite much memory - 
> but that could be fixed by also committing when the size of the temporary 
> files becomes too high. 

Indeed, having large diff sets trigger a commit should work nicely.

> > > If memory seems to dangerous, they could be in the (persistent) git
> > > staging area... I think my spider sense is tingling :)
> >
> > What do you mean by ``git staging area''?
> 
> In git we can add changes without committing them, and they are stored 
> somehow, so changing the file again doesn't change the commit until we do an 
> explicit "git add <file>". 
> 
> I don't really know how it works internally, but I think it should be 
> possible 
> to use it as shortterm-change-storage. 

Ah, you are probably talking about the index.  That's an interesting
idea :-) Though I'd opt for using some temporary branches rather than
the index.  This shouldn't be much slower, but will allow branch
switches (although I cannot tell now whether this is necessary at all
:-) ).
 
> > > The only problem would be that the necessary diskspace for the base
> > > system would be more than doubled. With todays 1TiB disks this shouldn't
> > > hurt too much, though. Userdata could still be in simple user-dirs (for
> > > optimization).
> >
> > I think this is trivially implementing by the usual technique of
> > setting / and /home on separate partitions.
> 
> ouch, seems I suffered from the "why take the simple path, when we can choose 
> the hard one"-disease ;-) 

So, I'm not the only guy suffering from this? ;-)
 
> > I'd rather doubt that: the package manager will have to store
> > snapshots anyway, and snapshots will consume space, too.  
> 
> The package manager just needs to keep lists of files, so it can notify a 
> filterfs which files to hide. Everything else is in the filesystem. 

In this case the package manager doesn't store the modifications.  It
can only hide or show files, which allows the user to do only the rm
operation, since creating a new configuration file may be, actually, a
modification of the base one.  What should happen to customized system
(configuration) files in this case?
 
> > Also, I'm
> > inclined to think that it is much easier to tune a VCS to consume less
> > space than to make a package manager work like a VCS.
> 
> The only way I know to avoid doubling the diskspace (for mostly 
> incompressible 
> files) would be to retrieve the files live from the VCS on each access, and I 
> fear that's quite slow compared to traditional fielsystems. 

Indeed, I somehow forgot about this detail.  Anyway, I still cannot
see how a package manager could store modifications in files.  I think
the only way would be to store the user modified files somewhere and
make them cover the base files when setting mine.  (I think this is
what your are suggesting and it's only now that I understand it :-( )
 
> > I'm not sure whether it will save *space* (you anyway need to store
> > some snapshots for each user), but it will definitely save resources,
> > since no special virtualization software is necessary.
> 
> You can safe space when users only do few modifications to the base system. 

Hm... I wonder how often the situation when a user does few
modifications to system configuration files happens.  If we intend to
make the user feel at home by using translator mine, it's very likely
that they would like modify quite a lot in their system.
 
> > OTOH, note that such a technique is also less secure: if a user does
> > some nasty thing (like a fork bomb), they bring down the whole system,
> > with all other users' environments.
> 
> I think that's why the Hurd needs a resource accounting framework like the 
> one 
> of viengoos. With that in place a user shouldn't be able to disturb the other 
> users experience anymore. 
> 
> If I understand it correctly, that is. 

Indeed, a proper resource accounting framework might result in a
complete isolation of different users.  So, let's port the Hurd to
Viengoos ;-)
 
> > > size_pkg_man = base_size + all changed files
> > > size_vcs = base_size + base_history + all diffs (user-history)
> >
> > Could you please explain what ``base_history'' is?
> 
> It the size of the base files +  all their history. 
> 
> > If you refer to the fact that doing a commit to VCS produces a new
> > history entry, then I'd say that could do commits ``in place'', that
> > is, undoing the top commit first, and then committing the new version
> > (I hope I'm making myself clear).  In this case size_vcs = base_size +
> > all changed files , just like size_pkg_man.
> 
> That would mean, we wouldn't be able to use simple branches for the users 
> anymore. 
> 
> A change in the base system should propagate to the branches, and when we 
> change the base, the diffs of the users don't fit anymore. That's why I think 
> that we won't be able to easily avoid carrying the history with us (though we 
> could cleanup from time to timeby saving per-user snapshots of the whole 
> system (the environment they see) and diffing these and the base system to 
> create new diffs). 

Well, updates in the base system are a serious problem whatever
mechanism of storing user data we choose.  The most fundamental issue
is that use configuration files are bound to become outdated, if the
user does not update the explicitly (and we aren't allowed to expect
the user to do that), so it is anyway necessary to update user
configuration files somehow.  I think that it is reasonable to allow
the user to solve merge conflicts via something like dispatch-conf in
Gentoo, but this is pretty orthogonal with the storage method: this
approach could be implemented in both VCS and package manager
approaches.
 
> > You're right :-) Let's hope that the dreams will come true (and work
> > on bringing them to reality in the meantime ;-) )
> 
> I do. 
> 
> If you like prosa, you can read one of my dreams on my website: 
> http://draketo.de/english/free-culture/light/myth-the-origin-of-the-polar-
> skulk
> 
> Besides: Did you see the FSF petition to free the Amazon kindle of DRM? 
> - http://www.defectivebydesign.org/amazon1984

I'll read these links today.  Sorry for not having read them so far --
it's lack of time, as usual :-(

Regards,
scolobb