[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Embedding libltdl into middleware
Re: Embedding libltdl into middleware
Tue, 3 Feb 2009 07:26:24 -0500
On Jan 31, 2009, at 9:26 AM, Ralf Wildenhues wrote:
Would the Libtool project consider adding a configure-time option to
prefix all public libltdlc symbols with a specified string? This
enable an embedded libltdlc to never conflict with an external
even if both were linked together into the same executable.
I don't think that would be sufficient, at least not in general.
why: libltdl modifies process-global state, namely the set of
modules. Even if we rename all libltdl-wide symbols, I think we could
at most guarantee that dlpreopened modules worked fine, but not
ones. Example: the Open MPI-embedded libltdl dlopens modfoo, and the
other one does so, too. One of the dlcloses it, the other one
On GNU/Linux, the dlclose only actually closes the module when the
reference to it is gone (i.e., dlopen uses refcounting), but I'm not
sure that this is the case with all module loaders which ltdl
Somebody would have to do research on this point.
FWIW, I think that this would be a reasonable limitation. If multiple
software layers are using their own copies of libltdl for whatever
reason, I think it is a fairly safe assumption that they will be
opening and closing different DLL's.
As a more practical matter, I'm pretty sure ATM there is code in ltdl
which assumes that we are the only ltdl operating (one example is the
preopening data structure for module symbols).
Global data symbols would have to be prefixed as well. If *all*
symbols are prefixed (functions and variables), you form a self-
contained set that a) are all consistent with each other, and b) will
not interact with a different internally-consistent set of ltdl symbols.
I cannot prove this
point without research, but also I am rather horrified at the prospect
of writing testsuite exposure for possible latent bugs which stem from
two included ltdl versions.
I'd be happy to write some test programs for these cases. After all,
this idea isn't worthwhile to me unless it actually works. ;-)
OTOH, libltdl is very much intended to be usable as both included and
external library. IOW, it should be possible to build Open MPI with
internal libltdl disabled.
Yes, we have a --disable-dlopen configure option for this case: OMPI
slurps all of its DLLs up into libopen-pal (and other libraries) and
does not build/embed libltdl. But it's not the default. We do not
have a mode to use an external ltdl (see below).
Hence, you could get into a scenario of having multiple OMPI's
installed just to support apps that explicitly use libltdl and apps
that do not. That doesn't seem desirable (at least from my
perspective ;-) ). More specifically, the issue is that OMPI violates
the Law of Least Astonishment; from the user's perspective, some MPI
applications work with a default build of Open MPI and others fail
(potentially seg faulting). I'm not sure what the reporting user had
to do to track down that ltdl was their particular cause of failure,
but I'm sure it cost them quite a bit of time to find that there was
basically a system-level conflict (vs. a bug in their code).
Here's another way of looking at it: why should users have to know/
care a) that OMPI is using ltdl, and b) exactly which version is being
used? My $0.02 is that these two facts should be completely hidden by
OMPI (IMHO). Indeed, OMPI applications can use libevent and PLPA and
not know/care that OMPI has embedded copies of them. Since both
libevent and PLPA are fully symbol-prefixed in OMPI, even if the
external libevent or PLPA is a different version than OMPI's embedded
version, they simply won't interact with each other and therefore the
possibilities of conflicts/problems are significantly diminished.
Of course, this will require the external
one to be new enough for Open MPI (and the other libraries') needs,
probably some hackery of the toplevel configure.ac (to support
--without-included-ltdl seamlessly). Have you tried to go this way?
I may be able to help with this.
FWIW, we previously explicitly rejected this route. IIRC, at the
time, we had a problem with Linux distros including old versions of
Libtool and other OS's not including Libtool at all. We wanted to
embed ltdl in Open MPI to avoid this dependency on the user's OS -- we
viewed ltdl as a small, simple tool that was easily embeddable and
could therefore eliminate a potential installation complication (e.g.,
we would *NOT* want to force users to upgrade their default Libtool
for fear of breaking other things on their system!). Specifically, we
agree with many of the points in the LT docs about why embedding ltdl
is a Good Thing.
Another example: OMPI is now using the ld_dlopenadvise() function
which was introduced in LT 2.something, and there are OS's out there
that still install LT 1.5.x.
To sum up: I think that prefix-embedded solution is certainly possible
and viable, even if it does have some limitations (e.g., as you cited,
two different ltdl's operating on the same DLLs may cause problems).
But it's still an improvement... :-)
|[Prev in Thread]
||[Next in Thread]|
- Re: Embedding libltdl into middleware,
Jeff Squyres <=