Re: Hiding nodes with unionmount

[Sergiu Ivanov]

Hello,

On Mon, Aug 03, 2009 at 11:14:32AM +0200, Arne Babenhauserheide wrote:
> Am Montag, 3. August 2009 07:25:38 schrieb Sergiu Ivanov:
> > I suppose this may work if you have the rights to write chroot/ , but,
> > unfourtunately, I can't check it.  For some reason, when I try chroot
> > <dir> /bin/bash , it tells me that it cannot find /bin/bash , although
> > this executable definitely exists :-(
> 
> Does it also exist in the chroot? 
> 
> You need chroot/bin/bash, I think. 

I tried putting bash into the chroot environment, but it didn't help.
Also, I tried invoking chroot only with the new root directory (since
the man page promises it should run the default shell), but it didn't
work either.  I guess I'm doing something wrong, so I'll ask tomorrow
at the meeting :-)
 
> > My understanding is that a sub-Hurd is something like another instance
> > of Hurd running on top of the same instance of gnumach.  It seems that
> > a sub-Hurd may reuse some servers from the main Hurd.  I'm not really
> > expert in this area, so you will probably like to read this:
> > http://www.gnu.org/software/hurd/hurd/subhurd.html .
> 
> As soon as the "subhurd needs root" issue is solved, it should allow every 
> normal user to boot his own system. 

AFAIK, Zheng has made some progress in this area.
 
> And it would be nice to be able to just use the same base Hurd for the main 
> hurd and all subhurds - it needs a readonly mounted partition, so it might be 
> possible to use the same partition for different subhurds and unionmount the 
> specific files into it. 
> 
> I don't know enough about the details to be able to judge if this would 
> really 
> be possible. 

You'd better ask antrik or Zheng.  They seem to posses the required
information.
> 
> > > We could then check which file we removed, and if the underlying
> > > filesystem changed, we'd be able to see the diffs and "merge" the changes
> > > with our changes.
> >
> > So, you are talking about something like branches?  The master branch
> > being the original / and each user having their own branch, right?
> 
> Jupp. It's only a strange idea, but I kinda like it :) 
> 
> We should be careful with talking about that, though: VCS filesystems have 
> been ridiculed almost as efficiently as the Hurd :) 

I hope we will not be banned from this list for such talks :-)

Frankly speaking, I didn't even know about VCS filesystems, to tell
nothing of their being ridiculed :-(
 
> The difference here would be that we could define fixed steps for committing, 
> for example logout. 
> 
> It could even mean that the system isn't saved till logout. All changes could 
> be in memory, until we logout and commit. 

Sounds reasonable for desktop use in regions where power shortages are
rare.
 
> If memory seems to dangerous, they could be in the (persistent) git staging 
> area... I think my spider sense is tingling :) 

What do you mean by ``git staging area''? 

(You must feel comfortable at a computer by having eight extremities
;-) )
 
> The only problem would be that the necessary diskspace for the base system 
> would be more than doubled. With todays 1TiB disks this shouldn't hurt too 
> much, though. Userdata could still be in simple user-dirs (for optimization). 

I think this is trivially implementing by the usual technique of
setting / and /home on separate partitions.
 
> > This sounds very interesting :-) I had the idea of employing git (for
> > instance) in managing my ~/ , but it makes little sense.  Multi-user
> > management is much more interesting :-)
> 
> :) 
> 
> I manage most of my writing projects with Mercurial, one dir per project, and 
> since that's quite convenient, I think git for the home partition isn't a bad 
> option either. 

I'm afraid I'll once try to do something like this :-) I think that
would be exciting :-)
 
> It also massively simplified backups. 
> 
> And I'm thinking of moving towards subrepos to make a backup as simple as 
> 
> $ hg clone ~/ backup # initial
> $ hg pull # incremental update

Cool idea, I'll keep this in mind :-)
 
> Just remember to exclude the Movies ;) 

Keep them on a separate Movie Partition :-D
 
> > Hm...  I'm not sure a package manager could replace a VCS: what it you
> > *modify* some files in /etc/ ? AFAIK, package managers are not
> > interested in what the installed files contain, so I'd say that a VCS
> > is required anyway.
> 
> When you modify you'd just save a snapshot which overwritees the file. 
> 
> The package manager would just supply the list of already existing files, and 
> the list of files which correspond to packages the user updated. 
> 
> - me: "Hey, translator, please setup my system."
> - translator: "Hey portage, which packages do you have installed?"
> - portage lists packages
> - translator: "And which of these packages do we already have in the base 
> system; never mind the versions?" 
> - portage lists packages
> - translator: "give me the corresponding files, please."
> - portage lists files
> - translator filters out the given files and unionmounts the locally 
> installed 
> replacements. 
> 
> Since portage knows about all files it installed itself, t also knows about 
> most config files. 
> 
> But maybe this would take more logic than a real VCS... 

Hm...  This sounds interesting, but also looks like the package
manager is required to perform the functions of a real VCS.  AIUI, you
would like the package manager to actually store snapshots of files,
which is very much like a VCS.  In other words, the package manager
would need to be tuned to work like a VCS; and then, why not simply
use a VCS already?
 
> The main advantage of using the package manager is that it doesn't double the 
> required diskspace. 

I'd rather doubt that: the package manager will have to store
snapshots anyway, and snapshots will consume space, too.  Also, I'm
inclined to think that it is much easier to tune a VCS to consume less
space than to make a package manager work like a VCS.
 
> > I still cannot see how a package manager can do that.  I'm setting up
> > Gentoo now, and I'm reading the Portage chapter of the handbook, but
> > it doesn't really help in understanding how it could track files.
> > Could you please explain how it should work?
> 
> I can give you the approximate commands (requires the currently masked 
> portage 
> 2.2): 
> 
> # in the base system
> $ emerge -p @installed > base # gives all installed packages
> 
> # in the user-system (before _logout_)
> $ emerge -p @installed > user # gives all programs of the user environment
> $ diff -u base user > diff # gives all packages which differ in the user env
> 
> # get a list of packages which are different in user from those in base
> $ cat diff | (some sed magic) > hidepkg  
> 
> # Get the files
> for i in `cat hidepkg | xargs`; do equery files $i >> hidefiles ; done
> 
> $ (Filter all files in hidefiles from the user environment)

Aha, I see :-) That's what I was expecting.  This sequence of commands
is something which should definitely take place in the environment we
are talking about, but I'd rather keep file management itself for the
VCS, while expecting of the package manager to do package management.
 
> > I think the possibility of this is implied by design: every
> > modification of a system-wide configuration file is stored as a diff
> > and is not applied to the *real* file.  So you can do whatever you
> > want in your environment :-)
> 
> For massively-multi-user-setups I assume that this would save damn much space 
> compared to full virtualization. 

I'm not sure whether it will save *space* (you anyway need to store
some snapshots for each user), but it will definitely save resources,
since no special virtualization software is necessary.

OTOH, note that such a technique is also less secure: if a user does
some nasty thing (like a fork bomb), they bring down the whole system,
with all other users' environments.
 
> To use the package manager instead of a VCS would save space, as long as the 
> size of the changed files doesn't exceed the size of the system history + the 
> size of the diffs. 
> 
> size_pkg_man = base_size + all changed files
> size_vcs = base_size + base_history + all diffs (user-history)

Could you please explain what ``base_history'' is?

If you refer to the fact that doing a commit to VCS produces a new
history entry, then I'd say that could do commits ``in place'', that
is, undoing the top commit first, and then committing the new version
(I hope I'm making myself clear).  In this case size_vcs = base_size +
all changed files , just like size_pkg_man.
 
> > > A firewall could for example be a translator on the network device, so it
> > > would automatically be completely transparent, and an audio equalizer
> > > could be a translator on /dev/dsp
> >
> > You are suggesting very interesting translator use-cases :-) I think
> > they are pretty orthogonal with multi-user management via the mine
> > translator, so one could implement these already now (well, except for
> > /dev/dsp).
> 
> I'm simply dreaming - but sometimes dreams come true; just look at the huge 
> free software community. 
> 
> Though... would anyone back at MIT in the birthtime of GNU have even dared to 
> dream of the sheer size of todays free software community? :-)
 
You're right :-) Let's hope that the dreams will come true (and work
on bringing them to reality in the meantime ;-) )

Regards,
scolobb