Re: Embedding libltdl into middleware

[Jeff Squyres]

On Jan 31, 2009, at 9:26 AM, Ralf Wildenhues wrote:

> > Would the Libtool project consider adding a configure-time option to
> > prefix all public libltdlc symbols with a specified string?  This  
> > would
> > enable an embedded libltdlc to never conflict with an external  
> > libltdl,
> > even if both were linked together into the same executable.
>
> I don't think that would be sufficient, at least not in general.   
> Here's
> why: libltdl modifies process-global state, namely the set of  
> dlopen'ed
> modules.  Even if we rename all libltdl-wide symbols, I think we could
> at most guarantee that dlpreopened modules worked fine, but not  
> dlopened
> ones.  Example:  the Open MPI-embedded libltdl dlopens modfoo, and the
> other one does so, too.  One of the dlcloses it, the other one  
> doesn't.
> On GNU/Linux, the dlclose only actually closes the module when the  
> last
> reference to it is gone (i.e., dlopen uses refcounting), but I'm not  
> so
> sure that this is the case with all module loaders which ltdl  
> supports.
> Somebody would have to do research on this point.

FWIW, I think that this would be a reasonable limitation.  If multiple  
software layers are using their own copies of libltdl for whatever  
reason, I think it is a fairly safe assumption that they will be  
opening and closing different DLL's.

> As a more practical matter, I'm pretty sure ATM there is code in ltdl
> which assumes that we are the only ltdl operating (one example is the
> preopening data structure for module symbols).

Global data symbols would have to be prefixed as well.  If *all*  
symbols are prefixed (functions and variables), you form a self- 
contained set that a) are all consistent with each other, and b) will  
not interact with a different internally-consistent set of ltdl symbols.

> I cannot prove this
> point without research, but also I am rather horrified at the prospect
> of writing testsuite exposure for possible latent bugs which stem from
> two included ltdl versions.

I'd be happy to write some test programs for these cases.  After all,  
this idea isn't worthwhile to me unless it actually works.  ;-)

> OTOH, libltdl is very much intended to be usable as both included and
> external library.  IOW, it should be possible to build Open MPI with  
> its
> internal libltdl disabled.

Yes, we have a --disable-dlopen configure option for this case: OMPI  
slurps all of its DLLs up into libopen-pal (and other libraries) and  
does not build/embed libltdl.  But it's not the default.  We do not  
have a mode to use an external ltdl (see below).

Hence, you could get into a scenario of having multiple OMPI's  
installed just to support apps that explicitly use libltdl and apps  
that do not.  That doesn't seem desirable (at least from my  
perspective ;-) ).  More specifically, the issue is that OMPI violates  
the Law of Least Astonishment; from the user's perspective, some MPI  
applications work with a default build of Open MPI and others fail  
(potentially seg faulting).  I'm not sure what the reporting user had  
to do to track down that ltdl was their particular cause of failure,  
but I'm sure it cost them quite a bit of time to find that there was  
basically a system-level conflict (vs. a bug in their code).

Here's another way of looking at it: why should users have to know/ 
care a) that OMPI is using ltdl, and b) exactly which version is being  
used?  My $0.02 is that these two facts should be completely hidden by  
OMPI (IMHO).  Indeed, OMPI applications can use libevent and PLPA and  
not know/care that OMPI has embedded copies of them.  Since both  
libevent and PLPA are fully symbol-prefixed in OMPI, even if the  
external libevent or PLPA is a different version than OMPI's embedded  
version, they simply won't interact with each other and therefore the  
possibilities of conflicts/problems are significantly diminished.

> Of course, this will require the external
> one to be new enough for Open MPI (and the other libraries') needs,  
> and
> probably some hackery of the toplevel configure.ac (to support
> --without-included-ltdl seamlessly).  Have you tried to go this way?
> I may be able to help with this.

FWIW, we previously explicitly rejected this route.  IIRC, at the  
time, we had a problem with Linux distros including old versions of  
Libtool and other OS's not including Libtool at all.  We wanted to  
embed ltdl in Open MPI to avoid this dependency on the user's OS -- we  
viewed ltdl as a small, simple tool that was easily embeddable and  
could therefore eliminate a potential installation complication (e.g.,  
we would *NOT* want to force users to upgrade their default Libtool  
for fear of breaking other things on their system!).  Specifically, we  
agree with many of the points in the LT docs about why embedding ltdl  
is a Good Thing.

Another example: OMPI is now using the ld_dlopenadvise() function  
which was introduced in LT 2.something, and there are OS's out there  
that still install LT 1.5.x.

To sum up: I think that prefix-embedded solution is certainly possible  
and viable, even if it does have some limitations (e.g., as you cited,  
two different ltdl's operating on the same DLLs may cause problems).   
But it's still an improvement...  :-)

--
Jeff Squyres
Cisco Systems