Re: LDADD and linker options like --whole-archive

[Marc Alff]

Hi Stefan

Stefan Puiu wrote:

> Hi Marc,
>
> what can I say, on one hand you've made me curious about this option.
> We're also experiencing long linking times (well, nothing compared to
> the old project you mentioned), but still, 4 minutes for linking in
> one modified library is a bit much.
>  

:-)

Let me try to explain how the whole thing would work in your case :
Assumption is that today, you have something like this

Current :
======

noinst_LIBRARIES libfoo.a libbar.a etc

libfoo_a_SOURCES = foo1.cpp foo2.cpp foo3.cpp ...
libbar_a_SOURCES = bar1.cpp bar2.cpp ...

So, foo*.cpp are compiled and linked into a static library libfoo.a

All the "dozens" of libraries are later used when linking a final binary :

<my_binary>_LDADD = ./libfoo.a ./libbar.a etc


> Unfortunately, we're not using libtool yet.

That's fine.

The generated makefile will use libtool (for the CVS HEAD patch), it's 
relatively transparent
from the user point of view (it just needs to be there, and the 
bootstrap is like aclocal; autoconf; libtoolize; automake; )

If for some reason you can not use libtool, you can specify
libfoo_o_LDR = /bin/ld -r
but it's less portable


> Our libraries are all
> static, built with noinst_LIBRARIES=libsomelib.a etc.
>
>  
Proposed :
=======

Instead, the Makefile.am would look like this :

noinst_RELOCATABLES = libfoo.o libfar.o etc
libfoo_o_SOURCES = foo1.cpp foo2.cpp foo3.cpp ...
libbar_o_SOURCES = bar1.cpp bar2.cpp ...

The difference is that an **object** named libfoo.o is created,
instead of a static library.

Since libfoo.o is an object, you can repeat the process of partial linking :

noinst_RELOCATABLES = my_dozen_lib.o
my_dozen_lib_o_LDADD = libfoo.o libbar.o etc


> After reading most of the emails in the discussion, I'm wondering
> about the documented need to run 'ld -Ur' with C++ programs, and do
> that only once. You said somewhere that the software you develop is
> using CORBA, and that you don't need to call constructors through
> CORBA (but I guess you do use constructors in other code) - I'm not
> sure if this would work for us.
>
>  
You can safely ignore the blurb about my project, it was to explain that :
   main.cpp does not call new foo();
but instead
   main.cpp call: a function pointer, which calls: new foo(),
   main.cpp only calls virtual functions on an interface that class foo 
implements,
   main.cpp does not know anything about class foo
so that the linker does not know about the "not-so-obvious" dependency 
between main and constructor in foo.o,
and therefore was not including the code from foo in the final binary.

Long story short, I **do** use C++ and it works, and without the need 
for "ld -Ur".
My case is similar in the sense that relocatables are "noinst", and used 
to procude
a final binary which is distributed.
I dont fully understand the implications of "ld -Ur" (did not ran into 
it yet), but if it's causing trouble,
I will be very interested in investigating what's going on, and address 
that.

> Also, I'm having a bit of trouble understanding how these relocatable
> objects work - so, when you link those in the final binary, is it that
> *all* the code is linked?
All the code that is part of the relocatable is used,
you have to include the relocatable in the final link command as follows :

<my_binary>_LDADD = ./libfoo.o ./libbar.o etc

or even better

<my_binary>_LDADD = ./my_dozen_lib.o


> Because right now we have both libraries
> that have to be included as a whole, since they have code which is not
> explicitly referenced, but they are used anyway,
For these, use _RELOCATABLES, so all the code will be included

> and libraries that we
> only want to get useful stuff from, not everything.
Keep these as regular static libs : the linker will cherry pick what is 
needed only.

> I recently changed
> our build to separate these, because before somebody had just put
> everything under --whole-archive, and I managed to shrink the biggest
> binary by 10MB.
Sounds like a good thing.

> Or do you suggest that I would only make the
> --whole-archive libraries into relocatables?
>  

Not sure I understand the "Or", but i will say yes anyway :)

Yes, because by definition, relocatables are more bulky :
**all** the code (in the relocatable) is used, so you want to build a 
relocatable using only code that you **know**
will be needed and is not explicitly referenced, not use the whole world.

To see how things can work, I can suggest the following :

First, see if it links :

Link manually your code using :

ld -r -o libfoo.o foo1.o foo2.o ...

Inspect the content :
   size,
   nm libfoo.o | c++filt
to make sure these relocatables contain what you expect.

Link your binary with libfoo.o

If that works, then relocatables can solve your problem,
you now have to see the automake part :
use the automake patch to use the _RELOCATABLES syntax,
which will generate the proper link command for you.

I posted two patches, the second (CVS) is better.
If you need an automake tarball with the patch, ask and I can make one.

Also, keep in mind this enhancement is not part of the official automake 
release yet,
but I hope to convince Alexandre (and Ralf for a change in libtool) at 
some point,
and seeing if it solves your problem will work towards that :)

Let me know how I can assist you with all this.
If your code is public and available on the web, I can look at it and at 
the Makefile.am changes.

Hope this helps,
Marc Alff.