Re: libtool convenience libraries and C++ constructors

[Tim Janik]

On Thu, 9 Mar 2006, Ralf Wildenhues wrote:

> Tim,
>
> Could we continue this discussion on a mailing list?
> For example address@hidden

done.

> * Tim Janik wrote on Thu, Mar 09, 2006 at 10:41:37AM CET:
> > On Thu, 9 Mar 2006, Ralf Wildenhues wrote:
> > > * Tim Janik wrote on Wed, Mar 08, 2006 at 10:17:22PM CET:
> > > > i'm looking into getting libtool convenience libraries to work
> > > > with C++ constructors (using 1.5.6 currently), is there any information
> > > > about this you could point me at?
> > >
> > > Hmm.  We've done some work for better template support in the CVS
> > > version.  What's the issue with constructors, initialization order or
> > > are there more issues?  A problematic example would be useful to test.
> > > Which system and compiler are you using?
> >
> > hm, when i wrote this, i was already suspecting that you simply don't
> > support C++ convenience libraries at all.
>
> I certainly would not claim the support is perfect yet.  :-)
>
> > it's probably best if i
> > just give a run down of my problems and current solutions bit-by-bit:
>
> Thanks.  Only few questions due to time constraints now, will look in
> more detail later:
>
> > - i'm using automake-1.4
>
> That is ancient.  It'd be great to use something newer (1.9.6 is
> current).

the last time i tried to port beast to a newer automake version (guess
that was 1.7 or so), they were missing essential features that automake-1.4
has. i will try another move at some point, but beast is a large project
so i really need time and patience for that ;)

> > - i'm using libtool-1.5.6 from debian stable (i think that's a bit
> >   patched here and there, but in code portions unrelated to my
> >   problems at hand)
>
> ACK.  1.5.22 is current, but the difference is probably not related to
> the issues we are discussing here.

iirc, debian patches it up to not barf on version numbers >= 999, and it's
otherwise 1.5.6. being able to build/develop on debian stable is actually
what keeps me at this version currently.

> > - my first attempt was to use:
> >     birnet/Makefile.am:noinst_LTLIBRARY = libbirnet.la
> >     sfi/Makefile.am:noinst_LTLIBRARY = libsfi.la
> >     bse/Makefile.am:lib_LTLIBRARIES = libbse.la
> >     bse/Makefile.am:libbse_la_LIBADD = ../birnet/libbirnet.la
> >     ../sfi/libsfi.la
> >   this even works for only-C sources, or for C++ sources without static
> >   constructors. because "ar" is used however, if static C++ constructors
> >   are being added to multiple C++ files, only the constructors of *one*
> >   object file will be executed before main(), the rest is simply forgotten
> >   about.
>
> Could you be bothered to show a minimal example to expose the issue?

yes and no ;)
the problem is, ever since libtool introduced the "--tag" option requirement,
i can't get it to do anything directly. so short of setting up a full
fledged project with automake, autoconf, libtool, etc, i can just try to
manually replicate what libtool does, and show that it fails:

=== a.cc ===
#include <stdio.h>
static struct A { A() { dprintf (2, "A()\n"); } } a;
=== b.cc ===
#include <stdio.h>
static struct B { B() { dprintf (2, "B()\n"); } } b;
=== c.cc ===
#include <stdio.h>
static struct C { C() { dprintf (2, "C()\n"); } } c;
=== main.cc ===
int main() { return 0; }

first, these source files make up a valid static constructor testcase:
$ g++-3.4 -Wall a.cc b.cc c.cc main.cc && ./a.out
C()
B()
A()
$

for linking these into a .la library, libtool uses ar:
  g++-3.4 -Wall a.cc -c
  g++-3.4 -Wall b.cc -c
  g++-3.4 -Wall c.cc -c
  ar cru lib.a a.o b.o c.o && ranlib lib.a

this does not result in constructor invocations though:
$ g++-3.4 -Wall lib.a main.cc && ./a.out
$

while ld -r works as advertised (creates a partially linked library):
  g++-3.4 -Wall a.cc -c
  g++-3.4 -Wall b.cc -c
  g++-3.4 -Wall c.cc -c
  ld -r -o lib.o a.o b.o c.o
  g++-3.4 -Wall lib.o main.cc

$ ./a.out 
C()
B()
A()
$


let me know if you need further info here. having a way to invoke libtool
manually would greatly help in testing of course...

> > - from reading about, i figured i had to essentially use "ld -r" to
> > partially
> >   link C++ sources (and "ld -Ur" or the compiler for the final stage). then
>
> Where did you read this information?  I need to know more here.
> I would like to know why this is necessary and why it works.

i cought up ld -r/-Ur from random mails when googling, (on was from Alexandre
Olivia, suggesting to use "ld -r" to create convenience libraries).
the rest was really just two sources, ld(1) (quoting relevant sections):

      -r
       --relocatable
           Generate  relocatable  output---i.e.,  generate an output file that
           can in turn serve as input to ld.  This  is  often  called  partial
           linking.   As  a side effect, in environments that support standard
           Unix magic numbers, this option also sets the output  file's  magic
           number  to  "OMAGIC".  If this option is not specified, an absolute
           file is produced.  When linking C++ programs, this option will  not
           resolve references to constructors; to do that, use -Ur.

           When  an  input  file  does  not have the same format as the output
           file, partial linking is only supported if that input file does not
           contain any relocations.  Different output formats can have further
           restrictions; for example some "a.out"-based formats do not support
           partial linking with input files in other formats at all.

       -Ur For anything other than C++ programs, this option is equivalent  to
           -r: it generates relocatable output---i.e., an output file that can
           in turn serve as input to ld.  When linking C++ programs, -Ur  does
           resolve references to constructors, unlike -r.  It does not work to
           use -Ur on files that were themselves linked  with  -Ur;  once  the
           constructor  table  has been built, it cannot be added to.  Use -Ur
           only for the last partial link, and -r for the others.

and http://www.gnu.org/software/libtool/manual.html#C_002b_002b-libraries

  11.1 Writing libraries for C++
   Creating libraries of C++ code should be a fairly straightforward
   process, because its object files differ from C ones in only three
   ways:
    1. Because of name mangling, C++ libraries are only usable by the C++
       compiler that created them. This decision was made by the
       designers of C++ in order to protect users from conflicting
       implementations of features such as constructors, exception
       handling, and RTTI.
    2. On some systems, the C++ compiler must take special actions for
       the dynamic linker to run dynamic (i.e., run-time) initializers.
       This means that we should not call ld directly to link such
       libraries, and we should use the C++ compiler instead.
    3. C++ compilers will link some Standard C++ library in by default,
       but libtool does not know which are these libraries, so it cannot
       even run the inter-library dependence analyzer to check how to
       link it in. Therefore, running ld to link a C++ program or library
       is deemed to fail.
   Because of these three issues, Libtool has been designed to always use
   the C++ compiler to compile and link C++ programs and libraries. In
   some instances the main() function of a program must also be compiled
   with the C++ compiler for static C++ objects to be properly
   initialized.

> >   i read some more (libtool sources) to figure how to get libtool to use
> >   "ld -r"
> >   instead of ar. turns out that libtool already does this when libraries are
> >   built as .lo or .o files. unfortunately, automake pukes at lt-library
> >   names that don't end in .la ;(
> >   otherwise i could have s/.la/.lo/ for the above noinst_LTLIBRARY entries
> >   and be done.
>
> I'm aware of some suboptimal Libtool support here;  see [1] for example.
> But to be honest I've forgotten about any bugs Automake-1.4 may have had
> that are fixed in 1.9.6.
>
> > - so what i'm currently using is basically:
>
> Seems ok but is not too stable: it uses internal Automake information;
> also probably fixing libtool issues may break it (but I hope not).

thanks for your risk estimation.
i'm glad i could get the build setup to work at all ;)

> > - i'm not using .lo for the partial library names above, because while
> >   libtool will build .lo libraries correctly (similar to how it builds
> >   .o in the above), the actual binary code ends up in the .lo file.
> >   and the .lo file will libtool later refuse to link into another library,
> >   because the .lo file it just created "is not a valid libtool archive"
> >   (i.e. no shell script wrapper). that's, allthough libtool in fact
> >   created the file itself... ;)
>
> Yes, that's one of the bugs I'm aware of.

ah, good.

> Cheers,
> Ralf
>
> [1] http://tkd.kicks-ass.net/GnuLibtoolProject/RoadMap should come back
> online soon, hopefully..

well, there's at least a Feb25 version at:
http://www.google.com/search?q=cache:rbdXT99trO0J:tkd.kicks-ass.net/GnuLibtoolProject/RoadMap+GnuLibtoolProject+RoadMap


---
ciaoTJ