[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Various race condition in install mode.

From: Sam Varshavchik
Subject: Various race condition in install mode.
Date: Thu, 07 Oct 2004 22:28:49 -0400

I've almost tracked down the source of random link-install failures that unexpectedly blows up a long build cycle.

I'm using libtool 1.5.6 with automake 1.8.3 on Linux. I'm building a fairly extensive app that installs a bunch of shared libraries and executables. And it uses a bunch of convenience libraries along the way.

The build cycle runs on an SMP box, and I use the -j option with GNU make to build stuff in parallel. gmake's -j option causes gmake to kick off multiple build commnds as long as, according to Makefile rules, they don't depend on each other in any way.

I believe that I see at least one, and possibly two problems that result from this, as well as a third issue that needs to be clarified in libtool's documentation:


One problem seems to be related to the following code fragment in the libtool script:

        if test "$mode" = relink; then
$run eval '(cd $output_objdir && $rm ${realname}U && $mv $realname ${realname}U)' || exit $?

As far as I can figure this out, under certain conditions, during an install cycle (I think), libtool temporarily renames a shared library as, then does something. I also see this, elsewhere in libtool:

        # Restore the uninstalled library and exit
        if test "$mode" = relink; then
$run eval '(cd $output_objdir && $rm ${realname}T && $mv $realname ${realname}T && $mv "$realname"U $realname)' || exit $?
          exit $EXIT_SUCCESS

        # Create links to the real library.
        for linkname in $linknames; do
          if test "$realname" != "$linkname"; then
$show "(cd $output_objdir && $rm $linkname && $LN_S $realname $linkname)" $run eval '(cd $output_objdir && $rm $linkname && $LN_S $realname $linkname)' || exit $?

So there's some switcheroo going on here, with the shared library files bouncing back and forth.

Now, an automake-generated Makefile will have multiple targets that are dependend on the 'make install' target (that's simplyfing things a bit, but that's the general idea).

The bottom line is that gmake is going to kick off multiple 'install' targets at the same time, because as far as automake knows, the install action for various binaries and libraries don't have any dependencies on each other. An install just copies a file, and automake-generated makefile doesn't think that one file (be it a binary or a library) needs to have another file installed first.

So, you're going to have multiple invocations of libtool, in install mode, occuring at the same time. And each instance is going to monkey around with the shared library files, renaming them back and forth. I think that the race condition problem here should now appear to be quite obvious.

There used to be a similar problem with autoconf -- where config.status used scratch files with fixed names, and under certain conditions an automake-generated makefile could end up running multiple instance of config.status at the same time, so they ended up scribbling over each other's temporary files. autoconf eventually got fixed by embedding the process id into temporary file names.

It looks like some form of locking is needed in install mode, so that multiple libtool installs are not going to stomp over each other's shared libraries.

One postscriptum to add is that I do not actually specify the -j option to gmake explicitly, I place the -j option in the MAKEFLAGS environment variable, so that each instance of a recursively-invoked gmake is going to run parallel stuff.


Here's an automake-generate rule that depends on the overall 'make install target':

install-exec-am: install-pkglibLTLIBRARIES install-pkglibexecPROGRAMS \
       install-pkglibexecSCRIPTS install-sbinPROGRAMS \

As we see, there are two sub-targets here of interest: install-pkglibLTLIBRARIES and install-pkglibexecPROGRAMS. The first one installs the shared libraries, the second one installs binaries that potentially link against the shared libraries. Neither target depends on each other, they only both depend on their parent, the install-exec-am target. Since install-pkglibexecPROGRAMS does not depend on install-pkglibLTLIBRARIES, gmake doesn't see any reason why it can't run these targets in parallel. Conceivably gmake could end up installing one of the binaries before it finishes installing all the libraries.

From researching the first issue, I understand that libtool in certain
situations, will do linking during the install mode. I haven't yet confirmed whether libtool tries to link against the libraries in their final, installation directory that it expects to be there already, or it will use the copy in the .libs subdirectory. If, as I suspect, it's linking against the libraries in their final install directories, they may not be there yet if automake ends up running the install target for one of the binaries before it finishes running the install targets for the libraries.

The reason why I suspect libtool is trying to link against libraries in their final installation directory is based on the third issue:


From $(modules)

Then I have also:

libcourierauthcommon_la_DEPENDENCIES=$(commonlibs) [ more stuff here ]

The first thing make install does is this:

/bin/sh ./libtool --mode=install /usr/bin/install -c '' '/tmp/x/usr/lib64/courier-authlib/'
libtool: install: warning: relinking `'
(cd /home/mrsam/rpm/courier-authlib-0.50-1/BUILD/courier-authlib-0.50; /bin/sh ./libtool --quiet --mode=relink gcc -I/usr/include/mysql -I/usr/include -O2 -g -pipe -Wall -I.. -I./.. -o -rpath /usr/lib64/courier-authlib -Wl,--whole-archive -Wl,userdb/libshuserdb.a -Wl,numlib/libshnumlib.a -Wl,random128/libshrandom128.a -Wl,gdbmobj/libshgdbmobj.a -Wl,libshauth.a -Wl,rfc822/libshencode.a -Wl,libhmac/libshhmac.a -Wl,md5/libshmd5.a -Wl,sha1/libshsha1.a -Wl,--no-whole-archive -lcrypt -lgdbm -ldb -inst-prefix-dir /tmp/x)

Which results in:

/usr/bin/ld: cannot find -lcourierauth

'make install' goes through the libraries in their listed order, and tries to install libcourierauthcommon first, which fails because libtool relinks against, which hasn't been installed yet.

The first library has a dependency on the second one, but automake really doesn't think that it should install the second library before the first one; that's really a dependency that's introduced by libtool.

Therefore, it needs to be documented that with libtool, the order of the listed libraries in _LTLIBRARIES should agree with any dependencies they have between themselves.

Attachment: pgprKdT75bV7c.pgp
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]