[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: lt_dlopen fails on AIX

From: Perry Smith
Subject: Re: lt_dlopen fails on AIX
Date: Thu, 30 Aug 2012 08:29:40 -0500


I don't know much about libtool.  I've built open source packages for AIX for 20+ years now.  The open source community has never written tools that 100% work with AIX.  Each package, like Apache, Ruby, Python, etc (from my perspective at least) has to coax the open source tools to working on AIX and they all do it their own different way.

Apache gets libtool to just make the .so files for its modules.  I don't know how it does that but I know it uses libtool and I know it produces .so files and not .a files for each of its loadable modules.

Ruby does not use libtool.  It does things itself using rake and other rubyisk methods.  It also produces .so files and does not put them into .a files.  Both of these are for what I will call "extensions".

You have found the two choices.  Either produce the .so file and use the dlopen call that works for other platforms or put the .so into a .a and then modify the dlopen call to add the extra flag and look inside the archive for the specific shared object.

Here are a couple clues that might help you decide which way to go.

A feature of AIX that the open source community doesn't seem to grok is that you can have 32 bit objects (shared or not shared) and 64 bit objects in the same archive and AIX's ld will pick out the proper size it needs at link time and also at load time… Really, the magic happens at link time.  So you can build a library in 32 bit mode and 64 bit mode, load them object files into a .a file and then link against it and not have two paths for the two versions.  Other platforms call this "fat".  You can do similar tricks with executables and loadable kernel extensions.

The benefit of this feature can be seen for what I will call "libraries"… objects that are automatically loaded at run time.  The selection happens at link time.  You can see this by doing "dump -H <executable>" and it will tell you which archives and objects within the archives will be used to run the executable.  (e.g. dump -H /bin/cat )  But that does not really have anything to do with dlopen.

dlopen, I would use the term, is used to open and load "extensions" -- as oppose to libraries.  The main point being that the dlopen is done under the application's control while loading libraries is done by ld and the loader.

Of topic slightly but… there are a ton of magic features that are available for selecting and loading libraries which the open source community doesn't grok because "its not like Linux".  I'm not 100% sure but from my perspective it appears that whatever the .la file does is mostly built into ld and the loader in AIX.  The .la files in AIX are not needed.

I don't know what "ltdl" is (which is what I think you are trying to get built but perhaps I misread your original post).  A did a tiny google search and didn't spot it.  From your description, it sounds like you want to just build the .so files and *not* put them inside a .a since you mention dlopen, etc.  I have no idea how to get libtool to do that but I know it can.  I save all my configure and make logs.  I can send you the log of the configure and build of Apache.  I just did it a month or so ago on 6.1 TL07 SP03.  Perhaps just pasting one of its calls to libtool would be enough.  I'm at home right now but I can do that here in a few hours if you think that might help.

By the way, I'm on this mailing list asking for help which never came.  I got one reply.  I don't even recall now what the question is.  I'm not complaining -- just describing that there are very few who work with open source code on AIX.

On that topic, the "prefix" set of tools I believe Ubunto (?) uses has been ported to AIX.  Or, at least, someone is making a valiant effort to do so.  You can google around and find that or I can point you to the place.  I don't use it because it is so heavy weight.  I have a bunch of scripts and gunk that I use.  My scripts will *not* help you with this specific problem because I haven't built the package you are trying to build.  But the prefix port might.

Sorry… its not Ubunto but Gento.

Mailling list archive is

I can see some AIX traffic in there.

Good luck,

On Aug 30, 2012, at 1:13 AM, Brice Goglin wrote:

I don't know how to use a debugger on AIX, but I added printf inside our embedded libltdl and found the following.

It looks at the ".la", parses it, finds the ".so" name, tries to dlopen it, and fails since this file is not installed. There's no way this code can work currently since it really expects a ".so" while only the ".a" and ".la" are installed.

Either I am missing a libtool flag when building/installing my plugin (a flag that would cause the ".so" to be installed). Or something broke between the way libtool and AIX build plugins.


Le 29/08/2012 16:06, Robert Boehne a écrit :
Step into the debugger and see what filename it's trying to use.  What I can tell you, is that 7 years ago all this worked on AIX 4.3, Linux, HPUX, OSF/1, IRIX, Solaris and Windows (at my previous employer).  We made C++ modules that were lt_dlopen'd at runtime.

On Wed, Aug 29, 2012 at 8:02 AM, Brice Goglin <address@hidden> wrote:
Le 29/08/2012 14:50, Robert Boehne a écrit :
Are you passing the name of the libtool "la" file to lt_dlopen?

I tried that (among others) but it fails with "file not found".
Is this the recommended strategy? I as initially using lt_dopenext on the libtool file without any extension.

Putting a shared object in an archive is the AIX way of building shared libs.
AIX also supports system V compatibility libraries that look more like
what you're used to on Linux & Solaris.
If you prefer to build these more familiar sorts, you can put -Wl,-brtl in LDFLAGS
which will cause Libtool to change the way it links, and create SysV-style libraries.

If dlopen is supposed to work on AIX with .la files pointing to .a files containing .so, that'd be fine. But it keeps failing here :/




On Wed, Aug 29, 2012 at 4:18 AM, Brice Goglin <address@hidden> wrote:

I cannot get ltdl to load plugins on AIX 6.1. I am compiling plugins
with -module -avoid-version. libtool generates a ".so" but then embeds
it inside a ".a" and it looks like it really wants me to use this ".a"
since the ".so" doesn't even get installed.

When lt_dlopen tries to open the plugins (tried dlopen with an extension
and dlopenext without extension), it fails saying "file not found". The
raw dlopen doesn't want to load the .a file because of a "bad magic number".

dlopen succeeds if I use "foo.a(" as a file name instead of
"foo.a", and if I add RTLD_MEMBER to the flags. But I don't see any
reference to RTLD_MEMBER in libltdl, so I assume I am not building the
plugins correctly.

What should I do to properly build my plugin? I assume I need to .la to
point to a .so instead of a .a but I can't find how do that. FWIW,
things work fine on Linux and Solaris, at least.

Here's the (dummy) plugin compile lines:

source='topology-fake.c' object='core_fake_la-topology-fake.lo' libtool=yes \
        DEPDIR=.deps depmode=xlc /bin/sh ../.././config/depcomp \
        /bin/sh ../libtool  --tag=CC   --mode=compile cc -qlanglvl=extc89 -DHAVE_CONFIG_H -I. -I../../src -I../include/private/autogen -I../include/hwloc/autogen  -I/pf/k/k203045/hwloc-1.6a1-1/build/include -I/pf/k/k203045/hwloc-1.6a1-1/include -DHWLOC_INSIDE_LIBHWLOC -DHWLOC_PLUGINS_DIR=\"/pf/k/k203045/hwloc-1.6a1-1/build/install/lib/hwloc\" -I../../src/libltdl -DHWLOC_BUILD_PLUGIN   -g -c -o core_fake_la-topology-fake.lo `test -f 'topology-fake.c' || echo '../../src/'`topology-fake.c
libtool: compile:  cc -qlanglvl=extc89 -DHAVE_CONFIG_H -I. -I../../src -I../include/private/autogen -I../include/hwloc/autogen -I/pf/k/k203045/hwloc-1.6a1-1/build/include -I/pf/k/k203045/hwloc-1.6a1-1/include -DHWLOC_INSIDE_LIBHWLOC -DHWLOC_PLUGINS_DIR=\"/pf/k/k203045/hwloc-1.6a1-1/build/install/lib/hwloc\" -I../../src/libltdl -DHWLOC_BUILD_PLUGIN -g -c ../../src/topology-fake.c -Wp,-qmakedep=gcc,-MF.deps/core_fake_la-topology-fake.TPlo  -DPIC -o .libs/core_fake_la-topology-fake.o
/bin/sh ../libtool  --tag=CC   --mode=link cc -qlanglvl=extc89  -g -module -avoid-version  -o -rpath /pf/k/k203045/hwloc-1.6a1-1/build/install/lib/hwloc core_fake_la-topology-fake.lo  -lpthread
libtool: link: /usr/bin/nm -B -BCpg  .libs/core_fake_la-topology-fake.o   | awk '{ if ((($ 2 == "T") || ($ 2 == "D") || ($ 2 == "B")) && (substr($ 3,1,1) != ".")) { print $ 3 } }' | sort -u > .libs/core_fake.exp
libtool: link: cc -qlanglvl=extc89 -Wl,-bM:SRE -o .libs/  .libs/core_fake_la-topology-fake.o   -lpthread -lc -Wl,-bnoentry  -qlanglvl=extc89   -Wl,-bE:.libs/core_fake.exp -Wl,-berok
libtool: link: ar cru .libs/core_fake.a .libs/
libtool: link: ( cd ".libs" && rm -f "" && ln -s "../" "" )

And the generated .la that lt_dlopen does not want to load:

cat src/
# - a libtool library file
# Generated by libtool (GNU libtool) 2.4.2 Debian-2.4.2-1.1
# Please DO NOT delete this file!
# It is necessary for linking the library.

# The name that we can dlopen(3).

# Names of this library.
library_names='core_fake.a core_fake.a'

# The name of the static archive.

# Linker flags that can not go in dependency_libs.

# Libraries that this one depends upon.
dependency_libs=' -lpthread'

# Names of additional weak libraries provided by this library

# Version information for core_fake.

# Is this an already installed library?

# Should we warn about portability when linking against -modules?

# Files to dlopen/dlpreopen

# Directory that this library needs to be installed in:

Thank you



reply via email to

[Prev in Thread] Current Thread [Next in Thread]