Re: Embedding libltdl into middleware

[Ralf Wildenhues]

Hi Jeff,

* Jeff Squyres wrote on Sat, Jan 31, 2009 at 01:50:09PM CET:
> A user has recently complained 
> (http://www.open-mpi.org/community/lists/users/2009/01/7793.php) that if 
> they link an MPI application against Open MPI v1.3's middleware libraries 
> and explicitly -lldtl on their link command line, Bad Things happen.  
> This is because the Open MPI-embedded libltldl is 2.2.6a and the 
> OS-installed version is 1.5.something.  This effect is described multiple 
> times in section 11.6 of the Libtool docs, so I assume the problem is 
> well-understood and I won't go into further detail.
>
> Would the Libtool project consider adding a configure-time option to  
> prefix all public libltdlc symbols with a specified string?  This would 
> enable an embedded libltdlc to never conflict with an external libltdl, 
> even if both were linked together into the same executable.

I don't think that would be sufficient, at least not in general.  Here's
why: libltdl modifies process-global state, namely the set of dlopen'ed
modules.  Even if we rename all libltdl-wide symbols, I think we could
at most guarantee that dlpreopened modules worked fine, but not dlopened
ones.  Example:  the Open MPI-embedded libltdl dlopens modfoo, and the
other one does so, too.  One of the dlcloses it, the other one doesn't.
On GNU/Linux, the dlclose only actually closes the module when the last
reference to it is gone (i.e., dlopen uses refcounting), but I'm not so
sure that this is the case with all module loaders which ltdl supports.
Somebody would have to do research on this point.

As a more practical matter, I'm pretty sure ATM there is code in ltdl
which assumes that we are the only ltdl operating (one example is the
preopening data structure for module symbols).  I cannot prove this
point without research, but also I am rather horrified at the prospect
of writing testsuite exposure for possible latent bugs which stem from
two included ltdl versions.

OTOH, libltdl is very much intended to be usable as both included and
external library.  IOW, it should be possible to build Open MPI with its
internal libltdl disabled.  Of course, this will require the external
one to be new enough for Open MPI (and the other libraries') needs, and
probably some hackery of the toplevel configure.ac (to support
--without-included-ltdl seamlessly).  Have you tried to go this way?
I may be able to help with this.

I realize that, since libltdl from Libtool 1.5.x and 2.2.x are not
completely compatible, there may be some churn in this.  That is however
a problem a distribution is regularly faced with in the face of
incompatible library updates: the implicit synchronization.  If there
are specific problems with this, feel free to post them here, and we
may be of help.

Cheers,
Ralf