Re: DESTDIR install and OpenBSD

[Ralf Wildenhues]

Hi Jacob,

* Jacob Meuser wrote on Tue, Jan 31, 2006 at 08:14:43AM CET:
> On Tue, Jan 31, 2006 at 04:33:10AM +0100, Ralf Wildenhues wrote:
> > * Jacob Meuser wrote on Fri, Jan 27, 2006 at 04:16:22AM CET:
> > > >
> > > >  If `$libdir/$linklib' happens to actually exist but
> > > > be the wrong thing, it fails to work around; 
> > > 
> > > true.  however, this is really close to the same problem as already
> > > exists wrt library search paths.  so it's not creating any more of
> > > a problem than already exists.

> > > will need to be reexamined soon though :)
> > 
> > Why?  Because we keep mumbling about not linking against indirect
> > deplibs?  ;-)
> 
> because hopefully libtool will soon instruct the linker to use correctly
> ordered library search paths.

Ah.  Are you implying that the ordering of the uninstalled paths has an
impact on this problem?  I previously thought it had not.  If there is
another ordering problem, I'd like to know about it.  Recent patches
have shown me once again that it is crucial to have good tests in our
test suite to prevent mistakes from returning.  They are hard, though:
I have not yet been able to come up with a system-independent version
of the `link-order.test' that exposes the very issue originally observed.

> > > > if it does not exist,
> > > > libtool fails to add hardcoding for the respective library, resulting in
> > > > execution startup failure.
> > > 
> > > it's not a problem on OpenBSD, because OpenBSD doesn't rely on the
> > > path being hardcoded.  in fact, it it preferred that the paths are
> > > not hardcoded.
> > 
> > I'd like this statement clarified a bit, to one (or more) of these:
> > - OpenBSD developers prefer to not have any hardcoded paths in libraries.
> > - OpenBSD developers prefer to not have any hardcoded paths in executables.
> 
> it is preferred that someone could install a binary package an any
> prefix, and have it work.  for example, the official packages are built
> with --prefix=/usr/local, but someone could install a binary openbsd
> package into /opt.

How would you make sure programs find their libraries and modules
automatically?  /usr/local/lib is neither in the default library search
path nor the default module (dlopen) search path.

I can understand the desire from a ports/distribution developer
standpoint very well, but from an end-user standpoint, I expect libtool
to cause my created programs to function when installed, without things
such as setting LD_LIBRARY_PATH, for example.

If OpenBSD were to desire not to link against indirect deplibs either,
it would be necessary to add DT_RPATH entries to libraries, too, not
just programs.

> > - OpenBSD compiler and/or linker and/or runtime linker are patched to
> >   not honor hardcoding arguments (--rpath).
> 
> it does honor --rpath

> it does honor hardcoded paths in libraries.  for example:
*snip*
> and it does honor hardcoded paths in binaries:  for example:
*snip*

Which?  The link editor or the runtime linker?  It's necessary to be
pedantic here.  My own observations are that both do:
- The link editor is basically some version of GNU binutils ld; it does
  honor rpaths for locating additional required libraries that are not
  listed on the command line.
- The runtime linker (ld.so) also obeys rpaths in libraries.

See this example:

mkdir sub
cd sub
echo 'int a() { return 0; }' > a.c
gcc -fPIC -c a.c
gcc -shared -o liba.so a.o
cd ..
echo 'extern int a(); int b() { return a(); }' > b.c
gcc -fPIC -c b.c
gcc -shared -o libb.so b.o -Lsub -la -Wl,-rpath,`pwd`/sub
echo 'extern int b(); int main() { return b(); }' > m.c
gcc -c m.c
# the next two lines show that ld looks for liba.so
# (and fails because we hide it):
mv sub sub-moved
gcc -o m m.o -L. -lb -Wl,-rpath,`pwd`
| /usr/bin/ld: warning: liba.so, needed by ./libb.so, not found (try using 
-rpath or -rpath-link)
| ./libb.so: undefined reference to `a'
| collect2: ld returned 1 exit status
# the next lines shows that ld finds liba.so through RPATH in libb.so:
mv sub-moved sub
gcc -o m m.o -L. -lb -Wl,-rpath,`pwd`
# the next line shows that ld.so finds liba.so:
./m; echo $?
| 0
objdump -p m | egrep 'NEEDED|RPATH'
|  NEEDED      libb.so
|  NEEDED      libc.so.38.4
|  RPATH       /tmp/test
objdump -p libb.so | egrep 'NEEDED|RPATH'
|  NEEDED      liba.so
|  RPATH       /tmp/test/sub

> > - the OpenBSD runtime linker does not honor hardcoded paths in libraries.
> > - the OpenBSD runtime linker does not honor hardcoded paths in libraries
> >   nor executable.
> 
> it does honor both.  the user should be able to do this if he so
> chooses.

OK.

> we don't want full hardcoded paths like /usr/local/lib/libfoo.so.x.y.
> RPATH is not such a big deal.

Ah!  I've never understood that it is paths in DT_NEEDED that you
dislike (as opposed to paths in DT_RPATH).  Well, yes, I can understand
this very well, now that I've finally understood this.  ;-)
Well, libtool currently simply does the wrong thing on OpenBSD then.
Let's fix this (self adds more notes to testsuite TODO list).

> keeping in mind that we want users to be able to install binary
> packages in any prefix, hardcoding paths is really never useful,
> at least as far as building packages goes.

Pending my question above.

> there may well be differences in the way we want things to work in
> our ports tree, and the way the average user would expect things
> to work.

Sure.  I understand the ports system has machinery in place to override
package's libtool scripts, so this is a solved problem (except for the
occasional nuisance that the innocent user will find that he cannot
reproduce a package build outside-of-ports to match the ports one;
luckily that is not too much of an issue ATM, so let's not worry about
it).

> > IIRC OpenBSD will not use the indirect deplibs issue because its runtime
> > linker does not load DT_NEEDED dependencies.  Right?
> 
> AFAICS, this is not currently true.  this may have been the case at
> one time, but there has been a bit of work to the runtime linker over
> the past few years.

Agreed.  See my test above which disproves this.

Cheers,
Ralf