relinking makes libtool link to the wrong library

[Simon Josefsson]

Hi!  We have received a bug report about some parts of recent gnutls
(libgnutls-extra.so.26) incorrectly links to the libgnutls.so in
/usr/lib/ rather than in $prefix, see original report:

http://lists.gnu.org/archive/html/gnutls-devel/2007-12/msg00038.html

To understand how this works, GnuTLS contains one core library
libgnutls.so (in lib/) and another library libgnutls-extra.so (in
libextra/).  The second library links to the former.

libextra/Makefile.am contains:

libgnutls_extra_la_LIBADD += ../lgl/liblgnu.la ../lib/libgnutls.la

The problem is easy to reproduce -- just build gnutls-2.2.0 locally on a
machine that has libgnutls.so.13 in /usr/lib -- but I don't know how to
solve it.  I'm posting some details here and hope someone can help me.

Make install in lib/ works fine, the core library is installed properly.
Libtool does not relink the libgnutls.so library.

Make install in libextra/ causes relinking to happen.  I think this is
expected, right?  The output is:

make[2]: Entering directory `/home/jas/gnutls-2.2.0/libextra'
test -z "/home/jas/gnutls-2.2.0/xxx/bin" || /bin/mkdir -p 
"/home/jas/gnutls-2.2.0/xxx/bin"
 /usr/bin/install -c 'libgnutls-extra-config' 
'/home/jas/gnutls-2.2.0/xxx/bin/libgnutls-extra-config'
test -z "/home/jas/gnutls-2.2.0/xxx/bin" || /bin/mkdir -p 
"/home/jas/gnutls-2.2.0/xxx/bin"
test -z "/home/jas/gnutls-2.2.0/xxx/lib" || /bin/mkdir -p 
"/home/jas/gnutls-2.2.0/xxx/lib"
 /bin/sh ../libtool --mode=install /usr/bin/install -c  'libgnutls-extra.la' 
'/home/jas/gnutls-2.2.0/xxx/lib/libgnutls-extra.la'
libtool: install: warning: relinking `libgnutls-extra.la'
(cd /home/jas/gnutls-2.2.0/libextra; /bin/sh ../libtool  --tag=CC --mode=relink 
gcc -std=gnu99 -g -O2 -D_REENTRANT -D_THREAD_SAFE -pipe -I/usr/local/include -g 
-O2 -D_REENTRANT -D_THREAD_SAFE -Wno-pointer-sign -no-undefined -lopencdk 
-version-info 27:1:1 -llzo2 -Wl,--version-script=./libgnutls-extra.vers -o 
libgnutls-extra.la -rpath /home/jas/gnutls-2.2.0/xxx/lib gnutls_extra.lo 
gnutls_openpgp.lo gnutls_ia.lo openpgp/libgnutls_openpgp.la ../lgl/liblgnu.la 
../lib/libgnutls.la )  
gcc -std=gnu99 -shared  .libs/gnutls_extra.o .libs/gnutls_openpgp.o 
.libs/gnutls_ia.o -Wl,--whole-archive openpgp/.libs/libgnutls_openpgp.a 
../lgl/.libs/liblgnu.a -Wl,--no-whole-archive  -Wl,--rpath 
-Wl,/home/jas/gnutls-2.2.0/xxx/lib -L/usr/lib -lopencdk -llzo2 -L/usr/local/lib 
-lgcrypt -lgpg-error -L/home/jas/gnutls-2.2.0/xxx/lib -lgnutls  
-Wl,--version-script=./libgnutls-extra.vers -Wl,-soname 
-Wl,libgnutls-extra.so.26 -o .libs/libgnutls-extra.so.26.1.1
/usr/bin/install -c .libs/libgnutls-extra.so.26.1.1T 
/home/jas/gnutls-2.2.0/xxx/lib/libgnutls-extra.so.26.1.1
(cd /home/jas/gnutls-2.2.0/xxx/lib && { ln -s -f libgnutls-extra.so.26.1.1 
libgnutls-extra.so.26 || { rm -f libgnutls-extra.so.26 && ln -s 
libgnutls-extra.so.26.1.1 libgnutls-extra.so.26; }; })
(cd /home/jas/gnutls-2.2.0/xxx/lib && { ln -s -f libgnutls-extra.so.26.1.1 
libgnutls-extra.so || { rm -f libgnutls-extra.so && ln -s 
libgnutls-extra.so.26.1.1 libgnutls-extra.so; }; })
/usr/bin/install -c .libs/libgnutls-extra.lai 
/home/jas/gnutls-2.2.0/xxx/lib/libgnutls-extra.la
/usr/bin/install -c .libs/libgnutls-extra.a 
/home/jas/gnutls-2.2.0/xxx/lib/libgnutls-extra.a
chmod 644 /home/jas/gnutls-2.2.0/xxx/lib/libgnutls-extra.a
ranlib /home/jas/gnutls-2.2.0/xxx/lib/libgnutls-extra.a
PATH="$PATH:/sbin" ldconfig -n /home/jas/gnutls-2.2.0/xxx/lib
----------------------------------------------------------------------
Libraries have been installed in:
   /home/jas/gnutls-2.2.0/xxx/lib
...

The libtool command is:

/bin/sh ../libtool  --tag=CC --mode=relink gcc -std=gnu99 -g -O2 -D_REENTRANT 
-D_THREAD_SAFE -pipe -I/usr/local/include -g -O2 -D_REENTRANT -D_THREAD_SAFE 
-Wno-pointer-sign -no-undefined -lopencdk -version-info 27:1:1 -llzo2 
-Wl,--version-script=./libgnutls-extra.vers -o libgnutls-extra.la -rpath 
/home/jas/gnutls-2.2.0/xxx/lib gnutls_extra.lo gnutls_openpgp.lo gnutls_ia.lo 
openpgp/libgnutls_openpgp.la ../lgl/liblgnu.la ../lib/libgnutls.la

That looks correct to me.

The gcc output is:

gcc -std=gnu99 -shared  .libs/gnutls_extra.o .libs/gnutls_openpgp.o 
.libs/gnutls_ia.o -Wl,--whole-archive openpgp/.libs/libgnutls_openpgp.a 
../lgl/.libs/liblgnu.a -Wl,--no-whole-archive  -Wl,--rpath 
-Wl,/home/jas/gnutls-2.2.0/xxx/lib -L/usr/lib -lopencdk -llzo2 -L/usr/local/lib 
-lgcrypt -lgpg-error -L/home/jas/gnutls-2.2.0/xxx/lib -lgnutls  
-Wl,--version-script=./libgnutls-extra.vers -Wl,-soname 
-Wl,libgnutls-extra.so.26 -o .libs/libgnutls-extra.so.26.1.1

This looks wrong to me, it contains:

   -Wl,--rpath -Wl,/home/jas/gnutls-2.2.0/xxx/lib ... \
   -L/usr/lib ... \
   -L/usr/local/lib ... \
   -L/home/jas/gnutls-2.2.0/xxx/lib -lgnutls

So the final -lgnutls will pick it up from /usr/lib instead.  I think
that the -L/usr/lib is bogus, that is the default library path anyway,
so it shouldn't be needed.  But I don't think fixing that will take care
of this issue -- the command line also contains -L/usr/local/lib, where
I also may have a older libgnutls.so.

Any ideas?

It seems libtool's re-ordering of -L and/or -l is causing problems here.

/Simon