Re: how to use circular dependencies

[Ralf Wildenhues]

Hi Tom,

* tom fogal wrote on Wed, Aug 10, 2005 at 09:52:07PM CEST:
> Ralf Wildenhues writes:
> > 
> >Convenience libraries are just collections of objects in an archive.
> >When another libtool-created library is linked against the archive, it
> >will be added as a whole, i.e., each object in the convenience lib will
> >be present in the output library (like the GNU ld --whole-archive
> >option).  When a program is linked against it, it will behave just as if
> >you link against a plain old archive -- each used symbol will be linked
> >in.
> >
> >Does this make matters clear?
> 
> This is a \emph{lot} clearer after looking up what --whole-archive does.
> Thank you. I think the --whole-archive reference would be a good thing
> to mention in the documentation. Maybe thats just my $0.02 though...

Ah, ok.  Would you be willing to update the documentation to this end?
(Be sure to work on the CVS HEAD or branch-2-0 copy of libtool.texi, if
you do.)

> >> >> address@hidden@lib/libSPPstrings.la
> >> >                     ^^
> >> >I believe there is a slash (/) missing here.
> >> 
> >> Its been too long for my little brain to remember clearly, but I
> >> vaguely remember having that slash there and seeing an extra / during
> >> builds. The build didn't seem to care about it, but it was
> >> aesthetically displeasing to me...
*snip*

> With the below as part of a Makefile.am:
> 
> ### /src/develop/Makefile.am snippet ###
> partrj_LDADD= \
>         ../ui/libParTrjUI.a \
>         ../grid/libSPPgrid.a \
>         ../models/libSPPModels.a \
>         ../share/libShare.a \
>         ../fields/libSPPFields.a \
>         ../menus/libParTrjMenu.a \
>         @address@hidden/strings/src/libSPPStrings.a \

You meant to write this here, right?
|         @top_builddir@/srcs/strings/src/libSPPStrings.a \


>         ../hash/libHash.a \
>         ../dbg/libSPPdbg.la \
>         ../t89files/libSPPt89.a \
>         ../models/libSPPModels.a
> ### /src/develop/Makefile.am snippet ###
> 
> I get the following build output:
> 
> ### '$ make' output snippet ####
> 
> Making all in develop
> make[3]: Entering directory
> `/home/tfogal/tmp/F77_Projects/srcs/develop'
> /bin/sh ../../libtool --mode=link g77 -Wall -I../include/
> 
> <snip -- bunch of object files + compiler options>
> 
> write_rkt.o write_rvt.o xyzstart.o ../ui/libParTrjUI.a
> ../grid/libSPPgrid.a ../models/libSPPModels.a ../share/libShare.a
> ../fields/libSPPFields.a ../menus/libParTrjMenu.a
> ../..//srcs/strings/src/libSPPStrings.a ../hash/libHash.a
> ../dbg/libSPPdbg.la ../t89files/libSPPt89.a ../models/libSPPModels.a
> 
> ### '$ make' output snippet ####
*snip*

> This is an autoconf + automake + libtool project, so its fully possible
> I'm screwing things up in a configure.in or something. 'grep' tells me
> I'm not doing anything totally braindead though, like overwriting it.

Let's see the following:
- the exact configure line you issued (copy and paste!)
- egrep 'top_(src|build)dir.=' Makefile
Do you modify any of the directory variables mentioned in
  info 'Preset Output Variables'
in your configure script?


> >> Okay. Would it suffice if I made a minimal project that demonstrates
> >> the behavior?
> >
> >That would still save a lot of work, yes.
> 
> Okay, took a bit longer than I anticipated but I got something that is
> much smaller and still demonstrates the issue. You can grab it at:
> 
> https://apollo.sr.unh.edu/~tfogal/circular.tar.gz

Thanks a lot!  It can be fixed without changing libtool, see below.

> It includes a 'makefile' and a 'makefile.working'. The former uses
> libtool and fails to link because libtool strips off a couple of
> libraries. The latter works fine, although it lists some libraries
> multiple times.
> 
> Its basically three directories (liba, libb, libc...) that are built
> into static libraries. liba needs a symbol from libc, libb needs
> symbols from both liba and libc, and libc needs symbols from both liba
> and libb.

First, naming a test library libc is, umm, "interesting" (as would be
libm, libl, liby, all of which you should avoid on POSIX systems unless
you provide their functionality, also libC -- often the C++ standard lib).
This prevents using "-Lpath/to/libc -lc", for example.

Second, the flag --preserve-dup-deps is used wrongly: Unfortunately, it
needs to go before $CC.  We should look into removing this limitation.

Now, there are a couple of ways to solve your problem IMHO, but they
depend on what you actually want.  This is the crux of the matter: it's
not clear.

In your example, you create a bunch of convenience archives which have
mutual (or circular, FWIW) dependencies.  We don't get to see whether
you want all of these libraries installed later (in which case they
should _not_ be convenience archives) or not.  Also, while creating the
libraries, you do not specify their dependencies (which obviously won't
work for all dependencies, since libtool currently does not provide a
way to specify dependency circles).

Scenario 1:
-----------
You don't actually want liba.a, libb.a libc.a installed.
Then there is a nice solution to your problem:
First create all of them as convenience archives, omit all dependencies
as you do now (convenience archives per definition don't have
dependencies; they can only be subsumed into other libraries).  Then
create a convenience archive consisting of the union of all of them:

  $LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -o libabc.la \
    liba/liba.la libb/libb.la libc/libc.la

and then link your program against libabc.la only.

You could even choose to want to install libabc, say to $libdir, then
you should add the option "-rpath $libdir", so libtool will not create a
convenience archive.

Independently, if you want libabc as static only, be sure to add -static
to creations of all four libraries, so the convenience archives don't
contain PIC code.  (Libtool currently has the limitation that it does
not build two convenience archives, one with PIC and one with non-PIC
code, when both types of libs are to be built.)


Scenario 2:
-----------
You actually want liba.a, libb.a, libc.a installed later (e.g., into
$libdir) but you do not under any circumstances want shared versions
of these (neither will your users). 

This is the troublesome part, and has two ways out:

(a) My preferred suggestion would be to just install one big archive, as
in Scenario 1.  That will be easier for your users as well (not having
to remember to link against all three).

(b) If that is not possible, here's a recipe that should work portably:

You unfortunately have to forego using .la files and the abstractions
they offer.  (Yes this is arguably a bug in Libtool.)

Create the .a archives directly like this (i.e., no .la files are
created):

 $LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -static -o liba/liba.a liba/*.lo
 $LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -static -o libb/libb.a libb/*.lo
 $LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -static -o libc/libc.a libc/*.lo

You _need_ to specify *.lo instead of *.o here.  (See the doc thread
running at the moment for why.)

Now, you can do

 $LIBTOOL --preserve-dup-deps --mode=link $CC $CFLAGS $LDFLAGS main.o \
        -o program liba/liba.a libb/libb.a libc/libc.a \
                   liba/liba.a libb/libb.a libc/libc.a

which turns into:
  gcc main.o -o program  liba/liba.a libb/libb.a libc/libc.a \
        liba/liba.a libb/libb.a libc/libc.a

Tadaa!  :)


Side notes:
- Strictly speaking, it can still be regarded as a libtool bug that the
  circular depencencies cannot be stored completely within the .la files
  (which would be yet another bug distinct from the one you reported).
  I'm not inclined to fix this one.

- Search for "FIXME: Pedantically, this is the right thing to do" in the
  libtool script and read both comments if you're interested for why
  this does not work in some cases and how to get around with a hack.

- If anyone feels some of this should be in the documentation as well,
  feel free to produce a patch to this end as well.  :)

> One thing that I found out when doing all of this is that it seems
> impossible to create circular dependency issues when the static archive
> is comprised solely of one object file. I had to split a library into
> multiple .c files, and thus of course multiple .o's, before I could
> produce a dependency problem.

Of course.  While the linker will only search for undefined symbols only
in archives _following_ on the link line, the set of symbols from all
objects already pulled are still available -- otherwise, you could end
up with duplicate symbols in the resulting program.  (This explanation
holds for static linking only!)

Everything clear now?

Cheers,
Ralf