Re: Bugreport: Incorrect forwarding of a shared library's -R flags when this library is linked to an executable

[Todd Gamblin]

Bob,

I first reported this issue on September 24, 2009.  I would post a link to the archive, but it looks like the archive is down (http://mail.gnu.org/mailman/listinfo/bug-libtool gives me 404 not found).  I've pasted my original email below, along with the initial response I got from Ralf.  I've been trying to get an answer on this since then, but Ralf hasn't responded to any of my followups.

It would be really nice if this were fixed, because at the moment I have to include the transitive closure of all my dependencies' -R arguments in my Makefiles to make things work properly.  I imagine that others trying to use this feature would have the same problem.

-Todd Gamblin

Here are the emails I'm referring to from September/October 2009:

From: Ralf Wildenhues <address@hidden>

Date: October 11, 2009 4:13:40 AM PDT

To: Todd Gamblin <address@hidden>

Cc: address@hidden

Subject: Re: Problem with -R and external libraries

* Todd Gamblin wrote on Thu, Sep 24, 2009 at 09:40:37PM CEST:

I'm having trouble getting -R to work as advertised.  From the

libtool docs:

-R libdir

If output-file is a program, add libdir to its run-time path. If

output-file is a library, add -Rlibdir to its dependency_libs, so

that, whenever the library is linked into a program, libdir will

be added to its run-time path.

However, when I link libraries that were built with -R into

executables, it doesn't look like rpath is getting set properly.

This kooks like a Libtool bug to me.  Especially looking at the output
of
 make check-local \
      TESTSUITEFLAGS='-k "Runpath in libtool library files" -v -d -x'

I vaguely remember being there before; possibly the obvious fix caused
some other problems, need to look.

Thanks,
Ralf

From: Todd Gamblin <address@hidden>

Date: September 24, 2009 12:40:37 PM PDT

To: address@hidden

Subject: Problem with -R and external libraries

I'm having trouble getting -R to work as advertised.  From the libtool docs:

-R libdir

If output-file is a program, add libdir to its run-time path. If output-file is a library, add -Rlibdir to its dependency_libs, so that, whenever the library is linked into a program, libdir will be added to its run-time path. 

However, when I link libraries that were built with -R into executables, it doesn't look like rpath is getting set properly.  Here's the complete description of the problem. 

I'm using libtool 2.2.6  on an AMD RedHat system (specifically Chaos).

I've got a tool, 'ef', that depends on a libtool library, 'libeffort.la'.  I build libeffort.la like this:

libeffort_la_SOURCES ...

libeffort_la_LIBADD = \

../callpath/libcallpath.la \

../libwavelet/libwavelet.la

libeffort_la_LDFLAGS = \

-avoid-version \

$(PAPI_LDFLAGS) $(PAPI_RPATH)

with the various variables set by configure like this (from the actual Makefile):

PAPI_LDFLAGS = -L/usr/local/tools/papi/lib64 -lpapi

PAPI_RPATH = -R /usr/local/tools/papi/lib64

libcallpath.la is built like this:

libcallpath_la_SOURCES = ...

libcallpath_la_LDFLAGS = \

  -avoid-version \

$(SW_LDFLAGS) $(SW_RPATH)

The flags there are set like this (again, by configure, so they're automatically generated and contain a few superfluous -L and -R's).

SW_LDFLAGS = -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -lstackwalk -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -lsymtabAPI -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -lcommon -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -lxml2 -L/usr/local/tools/libdwarf-2008.10.13/lib -ldwarf -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib/lib -lelf -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -liberty

SW_RPATH = -R /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -R /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -R /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -R /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -R /usr/local/tools/libdwarf-2008.10.13/lib -R /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib/lib -R /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib

libcallpath.la has dependencies like this:

dependency_libs=' -R/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -R/usr/local/tools/libdwarf-2008.10.13/lib -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -lstackwalk -lsymtabAPI -lcommon -lxml2 -L/usr/local/tools/libdwarf-2008.10.13/lib -ldwarf -lelf -liberty -lrt'

And libeffort.la has dependencies set like this:

dependency_libs='-R/usr/local/tools/papi/lib64 -R/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -R/usr/local/tools/libdwarf-2008.10.13/lib -R/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib/lib  -L/usr/local/tools/papi/lib64 -lpapi /g/g21/gamblin2/src/libra/chaos_4_x86_64_ib/callpath/libcallpath.la -L/g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib -L/usr/local/tools/libdwarf-2008.10.13/lib -lstackwalk -lsymtabAPI -lcommon -lxml2 -ldwarf -lelf -liberty /g/g21/gamblin2/src/libra/chaos_4_x86_64_ib/libwavelet/libwavelet.la -lrt'

Finally, I want to build an executable, ef, with these libs, but the rpath doesn't get set right.  Here's how I build ef:

ef_SOURCES=ef.C

ef_LDADD= libeffort.la 

But, when I run ldd on the executable, it doesn't know where any of the deps are, though the docs above lead me to believe it should:

(atlas1088):effort$ ldd .libs/ef

     libeffort.so => /g/g21/gamblin2/opt/libra/chaos_4_x86_64_ib/lib/libeffort.so (0x00002aaaaacc8000)

     libpapi.so.3 => not found

     libcallpath.so => /g/g21/gamblin2/opt/libra/chaos_4_x86_64_ib/lib/libcallpath.so (0x00002aaaaaf14000)

     libstackwalk.so => not found

     libsymtabAPI.so => not found

     libcommon.so => not found

     libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00002aaaab126000)

     libdwarf.so => not found

     libelf.so.1 => /usr/lib64/libelf.so.1 (0x00002aaaab463000)

     libwavelet.so => /g/g21/gamblin2/opt/libra/chaos_4_x86_64_ib/lib/libwavelet.so (0x00002aaaab677000)

     librt.so.1 => /lib64/librt.so.1 (0x00002aaaab8a7000)

     libmpich.so.1.0 => /usr/local/tools/mvapich-gnu/lib/shared/libmpich.so.1.0 (0x00002aaaabab0000)

     libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002aaaabd6d000)

     libm.so.6 => /lib64/libm.so.6 (0x00002aaaac06d000)

     libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002aaaac2f0000)

     libc.so.6 => /lib64/libc.so.6 (0x00002aaaac4ff000)

     libpapi.so.3 => /usr/local/tools/papi/lib64/libpapi.so.3 (0x00002aaaac855000)

     libstackwalk.so => /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib/libstackwalk.so (0x00002aaaaca8a000)

     libsymtabAPI.so => /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib/libsymtabAPI.so (0x00002aaaacd04000)

     libcommon.so => /g/g21/gamblin2/projects/dawn-sw/sw_beta5/x86_64-unknown-linux2.4/lib/libcommon.so (0x00002aaaad1f3000)

     libdwarf.so => /usr/local/tools/libdwarf-2008.10.13/lib/libdwarf.so (0x00002aaaad4a5000)

     libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaad6dc000)

     libz.so.1 => /usr/lib64/libz.so.1 (0x00002aaaad8e1000)

     libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaadaf5000)

     /lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)

     libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x00002aaaadd10000)

     libibumad.so.1 => /usr/lib64/libibumad.so.1 (0x00002aaaadf1d000)

     libperfctr.so.6 => /usr/local/tools/papi-3.6.1/lib/libperfctr.so.6 (0x00002aaaae122000)

     libpfm.so.3 => /usr/local/tools/papi-3.6.1/lib/libpfm.so.3 (0x00002aaaae332000)

     libibcommon.so.1 => /usr/lib64/libibcommon.so.1 (0x00002aaaae58b000)

And, indeed, the rpath isn't in the executable:

(atlas1088):effort$ readelf -a .libs/ef | grep RPATH

 0x000000000000000f (RPATH)              Library rpath: [/g/g21/gamblin2/opt/libra/chaos_4_x86_64_ib/lib:/usr/local/tools/mvapich-gnu/lib/shared]

I tried installing the whole package, as well, and the installed executable has the same problems.

Is this what's supposed to happen?  Am I missing something?  From the docs, it seems like  these executables should pick up the rpath of all their dependencies, but that isn't what's happening.  I could supply the -R args directly on the ef_LDFLAGS = line, but that's not really acceptable because i'd then have to remember what all the libraries and all their dependencies depend on, which can change.

I noticed things worked as I'd expect them to if I supply -rpath instead of -R, but the docs say that is for telling libtool the *install* directory of a library, and it won't let me install things if I do that.  I was excited briefly when things worked in my build directory, but then libtool refused to install with a message about the rpath directories.

Thanks,

-Todd

On Jan 6, 2010, at 9:35 AM, Stefan Muller wrote:

Bob,

Yes, it was me who introduced the exclamation marks to highlight the line I was talking about.

And yes, I have tested my programs with the official version of libtool and they do _NOT_ link correctly. In order to make them link correctly, I need to comment out this line in ltmain.sh.

Stefan

On Wed, Jan 6, 2010 at 9:29 AM, Bob Friesenhahn <address@hidden> wrote:

On Wed, 6 Jan 2010, Stefan Muller wrote:

Bob,
Thanks for your reply. However, I downloading the official FSF version of libtool from

http://*ftp.gnu.org/gnu/libtool/libtool-2.2.6b.tar.gz

is what I tried first. I compiled from sources and installed the official libtool in /usr/local,
which overrides the ubuntu installation in /usr. Below is the diff of the two versions of ltmain.sh -
both versions have the "-R*) ;;" line (#6002 and #6005, respectively), which makes my executables
link incorrectly.

Ahhh, I see.  I was confused by thinking that the comment with all the exclamation marks was in your copy.  It looked like a smoking gun, but was not.

Now that you have your hands on "official" versions, have you tested to see if your executables link correctly with the official versions?

Regardless of what OS maintainers do for their own purposes, it seems wise to always use official FSF versions for any package which gets distributed to a broader audience.

Bob
--
Bob Friesenhahn
address@hidden, http://*www.*simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://*www.*GraphicsMagick.org/

_______________________________________________
Bug-libtool mailing list
address@hidden
http://*lists.gnu.org/mailman/listinfo/bug-libtool