duplicate dependencies [was: RE: Very slow links: > 1 hour to lin k single executable - can we help?]

[Boehne, Robert]

Rob:

This is a known problem that results from supporting multiple
dependent archive libraries.  In some cases libtool can't strip
the redundant -lfoo -lbar in dependencies.  I would propose that
libtool would strip all duplicate dependencies (as it used to do)
unless a command line flag is given.  I did pose this question
a few weeks ago but didn't get any response.
Time permitting, I'll post a patch that would require the KDE developers
(and others in this situation) to add a flag that preserves all of
the dependencies when linking.  In my project, compile times go
from 6 hours to infinity.  ;)  So I've made my own ltmain.sh that
won't preserve any duplicate dependencies.  As soon as I can find
the time I'll work this into a patch.  In the mean time, anyone
care to suggest what the flag should be?  --preserve-dup-deps ???

Cheers!

Robert

-----Original Message-----
From: Rob Browning [mailto:address@hidden]
Sent: Wednesday, October 31, 2001 2:48 PM
To: address@hidden
Subject: Very slow links: > 1 hour to link single executable - can we
help?

(After looking at the past few months of this list, I didn't see any
 relevant posts, but if I just overlooked them, please let me know.)

With more recent libtool versions I've noticed suprisingly slow
libtool link times.  After some investigation, I discovered some gnome
related mail discussing the issue that suggested that libtool was
having problems with duplicate libs listed on the link line.

To try and fix this, I used the adapted snippet from gnome-config
listed below to safely remove duplicates from the link line.  This
worked, taking the compiles down from 20 minutes to 10 (this is on a
1GHz Athlon), and this was

However, since we're about to need to be able to lt_dlopen/dlsym most,
if not all of our libraries, I found that I needed to go back through
and make sure that all of our libs were fully inter-linked.  This
meant more libfoo.la links to each lib's libfoo_la_LIBADD.

After finishing all that, the compile times have jumped way past 20
minutes, to at least more than 90 minutes (as I compose this, the
compile still hasn't finished) -- It looks like duplicates are being
pulled in through the .la files themselves, and I there's no way for
me to clean that up before invoking libtool as I was with the
gnome-config hack.

If I edit our ./libtool file, and add an "set +x", I can see libtool
running some kind of recursive dependency analysis which over hundreds
of iterations builds the deplibs (I think) variable up to pages and
pages of duplicate libs.

If I strace the sh it's runing, I can see that it's trying to stat the
same library, say libm, in many locations, only to turn around and do
that all again.  Right now it has taken over an hour just trying to
link the final executable, and it's still not finished.

My questions:

  1) Is this expected behavior, or are we doing something wrong?
  2) If this is expected, are there plans to fix it?
  3) If there are plans, what can we do to help? :>

Oh, and right now, we're using libtool 1.4.2.

Thanks

#!/bin/sh

# This script removes any duplicate occurences of -lfoo in its args,
# but preserves the right ordering (last-duplicate-wins) for the
# linker.  Code originally from gnome-config.

all_flags="$@"

# Straight out any possible duplicates, but be careful to
# get `-lfoo -lbar -lbaz' for `-lfoo -lbaz -lbar -lbaz'
other_flags=
lib_L_flags=
rev_libs=
for i in $all_flags; do
    case "$i" in
    # a library, save it for later, in reverse order
    -l*|*.la) rev_libs="$i $rev_libs" ;;
    -L*|-R*)
        if $libs_L; then
            case " $lib_L_flags " in
            *\ $i\ *) ;;                        # already there
            *) lib_L_flags="$lib_L_flags $i" ;; # add it to output
            esac
        fi;;
    *)
        case " $other_flags " in
        *\ $i\ *) ;;                            # already there
        *) other_flags="$other_flags $i" ;;     # add it to output
        esac ;;
    esac
done

ord_libs=
if $libs_l; then
    for i in $rev_libs; do
        case " $ord_libs " in
        *\ $i\ *) ;;                    # already there
        *) ord_libs="$i $ord_libs" ;;   # add it to output in reverse order
        esac
    done
fi

echo $other_flags $lib_L_flags $ord_libs

exit 0

--
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

_______________________________________________
Libtool mailing list
address@hidden
http://mail.gnu.org/mailman/listinfo/libtool