Re: sysroot-support

[Ralf Wildenhues]

Hi Enrico,

Sorry about the late answer.

* Enrico Weigelt wrote on Sun, Jun 12, 2005 at 11:51:51PM CEST:
> * Ralf Wildenhues <address@hidden> schrieb:
> 
> > Is `SYSROOT' an arbitrarily chosen name, or does this have 
> > precedence somewhere?
> 
> it's a common name in cross compiling environments, ie.
> crosstool (http://www.kegel.com/crosstool/)

OK.  I must confess to know less than I'd like to about crosstool.

> > Not really, at least to me (sorry if to you it looks like I'm asking 
> > for the obvious).  SYSROOT is to facilitate cross-compiling resp. 
> > staged installs, right?  (More questions below.)
> 
> Using sysroot means that *everything* (besides the sourcetree)
> is used from some prefix. Maybe call it "build jail". All includes
> and all libraries to import live within the sysroot - nothing is 
> taken from the running system. 
> 
> Most of the sysroot work is done by a sysroot'ed toolchain, which
> is smart enough to look for includes, libs, etc from the right place.
> But other stuff, ie. the .la files, also has to be taken from 
> this place, and *not* from the running system. 
> 
> So for example, /usr/lib becomes $SYSROO/usr/lib

Well, for me here the most difficult problem is to know which part of
the tools already knows the sysroot and which not.  Only the compiler(s)
and the linker do?  And they do it for all of include paths, library
paths?  But they don't pretend to be in a jail, otherwise they could not
access the source tree, right?

One part of the difficulty here is that I believe there are more than
one (valid) approach, and this is just one of them.

More advanced questions: Let's assume libtool searches below $SYSROOT.
Let's assume it found some dependent library (candidate).
That one may have dependencies on its own, and maybe rpath info built
in.  Can we assume all of that belongs below $SYSROOT as well?  I'd
guess so.

Next step in this scenario:  We want to output the library.  It needs
rpaths.  We assumed all of that lives below $SYSROOT.  Now comes a part
which I don't understand with crosstool:  Would I hand it the paths
including $SYSROOT (with -L, with -R, with
/path/to/some/lib.a_or_lib.so)?  And also the real paths for finding the
dependent libs?  Or just rpaths with SYSROOT subtracted, and `ld' knows
what to do then?

By the way, how can I find out which paths the dynamic linker will
search by default?  Can the crosstool stuff actually help to execute the
built stuff as well (maybe by invoking an emulator for the host arch)?

Leads me to the next step: if we link an executable there, it should
have the SYSROOT stripped off eventual rpaths as well, naturally.
Will an eventual dynamic linker be able to put it back on if the user
wants to execute it while it's still visible under $SYSROOT (say, if we
are not cross-compiling)?

To make things a little more complicated: Libtool also allows execution 
of programs in $builddir.  They are temporarily linked against the
uninstalled libs under $builddir.  And maybe also against installed
libraries.  Whoops, now these need to have $SYSROOT _not_ stripped, so
they can be executed under $builddir.  Right?

See, I would love to implement a solution if we can come to some sort of
coherent answer.  Or maybe partial solutions.  But at least they must
not conflict with other types of uses which would break then.

> > Why would you only want to change paths below /usr/lib?  
> 
> No, of course not. It is just a dirty hack to get some packages
> (ie. x11-modular stuff) working. 

OK.

Then let's get to the next interesting issue: actual subtraction of
$SYSROOT (and, FWIW, other paths).

May I assume $SYSROOT already exists when configuring/building?
May I assume the directory structure below $SYSROOT already exists?

The first can easily be guaranteed, but the second cannot in general.
But that is bad.  Why?  Because in general I cannot do path computation
as soon as a `..' path component shows up.  Daniel Reed showed this
(on one of the libtool lists) when I had thought I already had the
answer -- a/b/../c != a/c  if b is a symlink to d/e -- all you can do is
try to `cd' into the former and hope that `pwd -L' gives you the right
answer (i.e., the one with $SYSROOT in front and not the mount point you
had $SYSROOT be a symlink to..).  Then again `pwd -L' is not portable..

You believe `..' does not show up?  Run `gcc --print-search-dirs'.

And then there are also the people that install software below symlinks.
As in /opt -> /home/opt.  But you would not want /home/opt to appear
anywhere (e.g. rpaths), because *that* is not how it will look like on
the hosts that nfs-mount this under /opt..

> I dont have enough in-deep knowledge of libtool internals for a really
> clean fix.

I am interested in the underlying problem.  After we have understood
that, we can see about libtool internals.

> The optimal approach would be simply adding $SYSROOT/ to the right 
> places where access to library files is actually done. 

Sure.  As said above, the question revolves around when exactly what
needs to be done.

> > Why do you trim double slashes?  
> 
> Also an ugly fix for better readable regex'es.

OK.  You probably wanted 's~//*~/~' though.

> > This is an honest question, I need to check but I believe this will 
> > potentiall blow up under win32 (where a leading double slash has a 
> > special meaning).
> 
> Uuh. win32 has an very diffent filesystem concept, and also many many
> things are completely different from the rest of the world. For a 
> really clean solution, we need either a completely unix-like environment
> to run libtool in or a separate win32 version.

Oh, sure.  I believe some support is possible.  It might suffice to
capsulate most of this in some shell functions or maybe a wrapper
script.  But no need to talk about this now, it's a different issue.

> <snip>
> > About the SYSROOT part of your patch: I would really love it if 
> > someone would invest the work and implement this right.  
> 
> Well, I'd love it, if someone would implement an strict and abstract 
> model of software's structure and *one* clean builder application 
> (per platform) instead of hacks for hacking hacks to create scripts 
> which somehow try to control the build process.

This is a different issue.  Or maybe just that I don't quite so agree
with this.

> I'm now trying to go there in smaller steps: first I'm implementing
> an abstraction of toolchain commands, so there's - from the view
> of normal packages - no more difference between various platforms,
> target systems, toolchains, etc.
> 
>     http://wiki.metux.de/public/Universal_Toolchain
> 
> You're really welcomed to contribute :)

Well, about that page let's just say I personally would not bash about
bugs in some software package without having written a bug report
against it first.

Regards,
Ralf