Autotest-test failures on AIX with NFS and shared libraries

[Ralf Wildenhues]

On AIX, shared libraries are peculiar in several ways.  One of the
"interesting" features is that their image is retained in core even
after the process using them has exited, presumably for efficiency.
They can be unloaded (by root) with the `slibclean' command, but not
by the regular user.

Now Libtool comes into play.  Its (Autotest) testsuite generates lots
of directories and shared libraries.  And tries to remove them.

Now comes some NFS version into play.  It keeps around .nfs* file
handles for removed files that are still in use, in the directory
where the file lives.  This file is not removable as long as used,
thus the directory not `rm -rf'-able.

Taken together, this results in weird-looking failures, where running
the same test twice can change the result, and lots of errors from `rm'
at the end of a test and at the start (if the test dir was nonempty).

This is bad.  It wouldn't be quite so bad when the user were informed
that there is something fishy going on.  Cf. also
<http://lists.gnu.org/archive/html/autoconf-patches/2006-03/msg00186.html>

OK to install this Autoconf patch, which warns, but deliberately only so
when removing fails before executing the test, not afterwards, and where
--clean reflects a nonzero exit status of rm to its callee?  Note that
exiting when the `rm' fails would prevent useful testing of Libtool on
AIX, and that warning again after the test seems like overkill to me.

Not sure if this should be documented in the section about Autotest.

I'm installing the Libtool patch below to work around the AIX issue.

Cheers,
Ralf

2007-02-12  Ralf Wildenhues  <address@hidden>

        * lib/autotest/general.m4 (AT_INIT): With --clean, return exit
        status of rm so we know when it failed.
        If cleaning of test dir failed before running the test, warn.
        Output the line separator in verbose mode before the warning
        to make clear the warning belongs to the following test.

Index: lib/autotest/general.m4
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/autotest/general.m4,v
retrieving revision 1.218
diff -u -r1.218 general.m4
--- lib/autotest/general.m4     25 Nov 2006 09:57:34 -0000      1.218
+++ lib/autotest/general.m4     12 Feb 2007 22:13:43 -0000
@@ -1,8 +1,8 @@
 # This file is part of Autoconf.                          -*- Autoconf -*-
 # M4 macros used in building test suites.
 
-# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006 Free Software
-# Foundation, Inc.
+# Copyright (C) 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007 Free
+# Software Foundation, Inc.
 
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -314,7 +314,7 @@
        test -d "$at_suite_dir" &&
          find "$at_suite_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
        rm -f -r "$at_suite_dir" "$at_suite_log"
-       exit 0
+       exit $?
        ;;
 
     --debug | -d )
@@ -706,6 +706,9 @@
   # Be sure to come back to the top test directory.
   cd "$at_suite_dir"
 
+  # Clearly separate the test groups when verbose.
+  test $at_group_count != 0 && $at_verbose echo
+
   case $at_group in
     banner-*)
       at_group_log=$at_suite_log
@@ -720,7 +723,8 @@
       at_group_log=$at_group_dir/$as_me.log
       if test -d "$at_group_dir"; then
        find "$at_group_dir" -type d ! -perm -700 -exec chmod u+rwx \{\} \;
-       rm -fr "$at_group_dir"
+       rm -fr "$at_group_dir" ||
+         AS_WARN([test directory could not be cleaned.])
       fi
       # Be tolerant if the above `rm' was not able to remove the directory.
       AS_MKDIR_P([$at_group_dir])
@@ -730,9 +734,6 @@
 
   echo 0 > "$at_status_file"
 
-  # Clearly separate the test groups when verbose.
-  test $at_group_count != 0 && $at_verbose echo
-
   # In verbose mode, append to the log file *and* show on
   # the standard output; in quiet mode only write to the log
   if test -z "$at_verbose"; then




2007-02-12  Ralf Wildenhues  <address@hidden>

        Fix spurious testsuite failures on AIX due on NFS mounts due to
        shared library images that cannot be removed without `slibclean'
        which needs superuser privileges.

        * tests/testsuite.at (PREPARE_TESTS): Define $mkdir_p globally.
        (LT_AT_MVDIR): New macro to emulate `mv dir dest'.
        * tests/destdir.at: Use LT_AT_MVDIR.
        * tests/shlibpath.at: Likewise.
        * tests/static.at: Likewise.

Index: tests/destdir.at
===================================================================
RCS file: /cvsroot/libtool/libtool/tests/destdir.at,v
retrieving revision 1.2
diff -u -r1.2 destdir.at
--- tests/destdir.at    6 Feb 2007 18:56:57 -0000       1.2
+++ tests/destdir.at    12 Feb 2007 22:17:35 -0000
@@ -33,7 +33,6 @@
 fi
 libdir=$prefix/lib
 bindir=$prefix/bin
-: ${mkdir_p="$abs_top_srcdir/libltdl/config/install-sh -d"}
 $mkdir_p src $prefix $DESTDIR $DESTDIR$prefix
 eval `$LIBTOOL --config | $EGREP '^(FGREP|OBJDUMP)='`
 ])
@@ -57,8 +56,8 @@
 AT_CHECK([$LIBTOOL --mode=install cp m $DESTDIR$bindir/m],
         [], [ignore], [ignore])
 $LIBTOOL --mode=clean rm -f liba.la m
-mv $DESTDIR$libdir $libdir
-mv $DESTDIR$bindir $bindir
+LT_AT_MVDIR(["$DESTDIR$libdir"], ["$libdir"])
+LT_AT_MVDIR(["$DESTDIR$bindir"], ["$bindir"])
 # We ignore failure on purpose here: we may not be root.
 AT_CHECK([$LIBTOOL --mode=finish $libdir], [ignore], [ignore], [ignore])
 LT_AT_EXEC_CHECK([$bindir/m])
@@ -96,9 +95,9 @@
 AT_CHECK([$LIBTOOL --mode=install cp m $DESTDIR$bindir/m],
         [], [ignore], [ignore])
 $LIBTOOL --mode=clean rm -f liba1dep.la liba2dep.la liba.la m
-mv $DESTDIR$libdir $libdir
-mv $DESTDIR${libdir}2 ${libdir}2
-mv $DESTDIR$bindir $bindir
+LT_AT_MVDIR(["$DESTDIR$libdir"], ["$libdir"])
+LT_AT_MVDIR(["$DESTDIR${libdir}2"], ["${libdir}2"])
+LT_AT_MVDIR(["$DESTDIR$bindir"], ["$bindir"])
 # We ignore failure on purpose here: we may not be root.
 AT_CHECK([$LIBTOOL --mode=finish $libdir ${libdir}2], [ignore], [ignore], 
[ignore])
 
Index: tests/shlibpath.at
===================================================================
RCS file: /cvsroot/libtool/libtool/tests/shlibpath.at,v
retrieving revision 1.1
diff -u -r1.1 shlibpath.at
--- tests/shlibpath.at  17 Sep 2006 19:26:28 -0000      1.1
+++ tests/shlibpath.at  12 Feb 2007 22:17:35 -0000
@@ -35,7 +35,7 @@
 AT_CHECK([$LIBTOOL --mode=link $CC $CFLAGS $LDFLAGS -o m2 m.$OBJEXT -Lsub/lib 
-la],
          [], [ignore], [ignore])
 
-mv sub moved
+LT_AT_MVDIR([sub], [moved])
 mkdir sub sub/lib sub/bin
 echo 'int wrong_lib () { return 0; }' > a.c
 $LIBTOOL --mode=compile $CC $CPPFLAGS $CFLAGS -c a.c
Index: tests/static.at
===================================================================
RCS file: /cvsroot/libtool/libtool/tests/static.at,v
retrieving revision 1.1
diff -u -r1.1 static.at
--- tests/static.at     20 Mar 2006 20:46:30 -0000      1.1
+++ tests/static.at     12 Feb 2007 22:17:35 -0000
@@ -100,18 +100,18 @@
 
 func_move_libs ()
 {
-  mv ${1} ${1}-moved
-  mv ${2} ${2}-moved
-  mv $srcdir_broken ${1}
-  mv $libdir_broken ${2}
+  LT_AT_MVDIR(["${1}"], ["${1}-moved"])
+  LT_AT_MVDIR(["${2}"], ["${2}-moved"])
+  LT_AT_MVDIR(["$srcdir_broken"], ["${1}"])
+  LT_AT_MVDIR(["$libdir_broken"], ["${2}"])
 }
 
 func_restore_libs ()
 {
-  mv ${2} $libdir_broken
-  mv ${1} $srcdir_broken
-  mv ${2}-moved ${2}
-  mv ${1}-moved ${1}
+  LT_AT_MVDIR(["${2}"], ["$libdir_broken"])
+  LT_AT_MVDIR(["${1}"], ["$srcdir_broken"])
+  LT_AT_MVDIR(["${2}-moved"], ["${2}"])
+  LT_AT_MVDIR(["${1}-moved"], ["${1}"])
 }
 
 # make sure the program can be run.
Index: tests/testsuite.at
===================================================================
RCS file: /cvsroot/libtool/libtool/tests/testsuite.at,v
retrieving revision 1.48
diff -u -r1.48 testsuite.at
--- tests/testsuite.at  12 Feb 2007 19:55:02 -0000      1.48
+++ tests/testsuite.at  12 Feb 2007 22:17:35 -0000
@@ -40,6 +40,7 @@
 else
   unset=false
 fi
+: ${mkdir_p="$abs_top_srcdir/libltdl/config/install-sh -d"}
 m4_divert_pop([PREPARE_TESTS])dnl
 
 
@@ -185,6 +186,21 @@
 ])
 
 
+# LT_AT_MVDIR(SRC, DEST)
+# ----------------------
+# Move contents of a directory recursively.
+# Avoid `mv DIR ...' due to issues with non-deletable shared libraries
+# on AIX on NFS mounts.  We cannot use slibclean as non-superuser.
+m4_define([LT_AT_MVDIR],
+[$mkdir_p $2
+(cd $1 && find . -type d) | (cd $2 && xargs $mkdir_p)
+for file in `cd $1 && find . -type f -print -o -type l -print`
+do
+  mv $1/$file $2/$file
+done
+rm -rf $1
+])
+
 ## ------------------------------- ##
 ## Files for a small ltdl project. ##
 ## ------------------------------- ##