RFC: indirect deplibs v2

[Ralf Wildenhues]

I haven't come as far as I would have liked (rpath discussion is very
incomplete still), and I won't be responding for the next couple of
days, but here's my current version of the RFC (pasted info plus texinfo
source attached) for you to tear apart.

Thanks for all your valuable input so far, more feedback greatly
appreciated.

Regards,
Ralf

10 Inter-library dependencies
*****************************

First, we define a set of terms to use later.

   A program or library has a "direct base library", if it depends on
some interface that base provides (see *Note Interfaces:: for a more
thorough description).
     If you use `cos' in your program or library, then `libm' is a direct
     base library, and your program or library is directly derived from it.
   The program or library depending on the base library will be called
"derived program or library", or short "derived object".  (1)

   An object has an "indirect base library", if it does not depend on
any interfaces of the library itself, but some base library of the
object depends on such an interface.
     A program makes no use of any math routines itself, but uses a
     library `libalgorithms', which uses the `cos' function.  Then
     `libm' will be a indirect base library of the program, but a
     direct library of `libalgorithms'.  Conversely, `libalgorithm'
     is directly derived from `libm', but your program is indirectly
     derived from `libm'.

   A library "exposes a base library" if it possibly causes a derived
library or program to rely on an interface of that base.
     In example above, if `libalgorithms' is to be used with a header
     file `algorithms.h', which contains
       `#define mathop(angle) cos(angle)'
     then the algorithms library exposes the math library.
   In general, the notion of exposure is not detectable automatically.
The following guide can be given:  Whenever library headers require
exposing external interfaces to sources of derived objects, for example
because they require inclusion of base library headers or include these
headers themselves, the base library should be considered exposed to
derived code.  These are not the only possibilities, however.

   A system with a "needed-following linker" has a means to record both
dependencies on other libraries based on the soname of the base library,
and rpaths in which those other libraries are to be searched.  It also
has a linker that picks up both the soname as well as the rpath
information to load all base libraries at runtime.  For example, the
Linux linker does all of this.  (2) (3)

   Before Libtool version 2.2, the handling of inter-library
dependencies has ignored the fact that some system linkers are smart
enough to figure out the base libraries of base libraries (recursively)
themselves, and always linked in all base libraries into the output.

   Whenever a lowlevel library changes its interface (soname), this
requires recompiling of all derived objects.  If most of the derived
objects do not make use of the lowlevel interfaces in any way, this is
suboptimal on systems with a needed-following linker.  It should
suffice to recompile the directly derived objects (which itself do not
change their interface).  On systems without needed-following linker,
recompilation of all derived objects is necessary.

   In order to allow for portable linking using libtool, there are a
few flags which can be added to the link line in order to specify the
dependency information: The available options are `-exposed-libs',
`-private-libs', `-direct-libs' and `-indirect-libs'.  These options
behave like two types of switches and hold for the base libraries
listed to the right of the option, until encounter of another switch of
the same type.

   Libtool and calculates from the information given on the command
line together with the information gathered from base libraries two
sets:  A set of libraries to be directly linked to, and a set of
libraries which are considered exposed to derived objects.  These sets
are then recorded in the output.  (4)

   Speaking in terms of the directed, generally not acyclic, graph
underlying inter-library dependencies: library dependency corresponds
to reachability along a path, direct dependency corresponds to a path
of length one, and exposure is an edge attribute.  The notion of a
"valid graph" then imposes certain connectivity conditions when
inserting new vertices (which corresponds to creating derived object),
namely:  directed paths of length two require a shortcut if the second
edge of the path has the exposure attribute.  (5)

   The semantics are as follows:

   * If none of the options are specified on the command line, all base
     libraries are considered direct and exposed.  Similarly, all bases
     picked up from unadorned libtool libraries (i.e., the information
     found in `.la' files lacking exposure or directness information)
     are treated as direct, exposed base libraries.  (6)

   * The switches can be given as often as necessary and hold for the
     following arguments (thus, order is important) until encounter of
     another switch of the same type.  (7) Other than the necessity of
     specifying libraries in order, the switches are pure logical
     entities denoting edges and edge attributes to be added to an
     existing graph.

   * Exposed bases of the object are always treated as direct bases,
     whether so denoted or not.  The line `libtool --mode=link $CC -o
     a.out libfoo.la -exposed-libs -indirect-libs libbar.la' will link
     direcly against both `libfoo.la' as well as `libbar.la'.

   * An exposed base of a direct base library is a direct base, and
     itself again exposed unless listed as private dependency.

   * If a library is mentioned (or picked up) as both direct and
     indirect base, it is treated as direct base.  Direct bases of
     direct bases are direct.

   * Otherwise, a base is indirect and not exposed.


   On systems with needed-following linker, only the direct base
libraries encoded in the output.  On all other systems (and of course
when linking statically), all bases are encoded in the output.

   A few examples are probably necessary:

   TODO: Insert good examples here.

   TODO: RPATH discussion!

   Libtool < 2.2: Programs break if you install any newer library
anywhere in the full rpath presented in program

   New semantics: Programs break if you install any newer or have a
library with matching soname anywhere in the union of all picked-up
rpaths and the set of all deplibs.

   TODO: Move the following to appropriate place.

   On systems without a needed-following linker, it is necessary to
explicitly list all base libraries, direct as well as indirect.
Libtool is smart enough to find all base libraries of libtool
libraries.  But often, base libraries are not produces by libtool on
such systems.  So portable linking needs to specify these indirect base
libraries.

   On systems with needed-following linker, users are frequently
tempted to not include indirect base libraries in the link line at all.
Be warned, however, that this is not portable, because it will
generally fail on other systems.  Only if your direct base libraries
are guaranteed to be libtool libraries (`.la' files), e.g. because they
are built as part of your software package as well, does omitting
indirect base libraries work portably.

     libltdl.la depends on libdlloader.la,
     libdlloader.la may depend on libdld, depending on the system.
     libltdl.la depends on libdlloader.la,
     libdlloader.la may depend on libdld, depending on the system.

   Since both `libltdl.la' and `libdlloader.la' are built within the
same package and thus guaranteed to both be libtool libraries, it is ok
to omit `-ldld' from the link line of `libltdl.la'.

   Open questions:
   * Is it possible to find out in all cases which things actually mean
     the same library?

   * Is it necessary for the previous question to be answered yes?
     Also on systems without needed-following linker?

   * Is this really backwards-compatible?  I think so, libtool-1.5 and
     newer seem to ignore `-indirect-libs' and `-direct-libs'.

   * Can it be implemented with reasonable effort?

   * What about rpath?

   ---------- Footnotes ----------

   (1) This discussion is not about legal aspects, and as such, we do
not use this notion with the meaning it has with respect to law.

   (2) prominent counter-example

   (3) Do I have to define soname?  Should I use a different word?

   (4) If the output is either a library itself or loadable, see (*note
Dlopened modules::).

   (5) This also means that libtool will possibly have to iterate over
the set of libraries to consider until a stable set of new attributed
edges is reached.

   (6) This is a safe, backward-compatible assumption, but the notion of
full exposure might be too pessimistic, at least for system libraries.

   (7) One may restate the first point as implicitly added
`-exposed-libs -direct-libs' at the beginning of the link line.

RFC.deplibs.texi
Description: TeXInfo document