[PATCH] posix-shell.m4: reject Solaris 11's /bin/sh (due to umask)

[Jim Meyering]

Hi Paul,

I discovered that Solaris 11's /bin/sh exhibits the following
surprising behavior:

    $ /bin/sh -c 'umask 22; (umask 0); umask'
    0000

That would cause test failures in coreutils, because gl_POSIX_SHELL
accepted /bin/sh.  The following change is enough to make
gl_POSIX_SHELL reject that shell and instead use /bin/bash, which
is also provided:

        posix-shell.m4: reject Solaris 11's /bin/sh (due to umask)
        * m4/posix-shell.m4 (gl_POSIX_SHELL): Reject a shell that
        lets a umask modification leak out of a subshell.  Otherwise,
        Solaris 11's /bin/sh would be accepted and thus cause unwarranted
        failures in coreutils test suite.

diff --git a/m4/posix-shell.m4 b/m4/posix-shell.m4
index 7f3b1e9..4c56193 100644
--- a/m4/posix-shell.m4
+++ b/m4/posix-shell.m4
@@ -31,12 +31,16 @@ AC_DEFUN([gl_POSIX_SHELL],
        func_ret_failure () {
         return 1
        }
+       subshell_umask_sanity () {
+        (umask 22; (umask 0); test $(umask) -eq 22)
+       }
        test "[$](echo foo)" = foo &&
        func_success &&
        ! func_failure &&
        func_ret_success &&
        ! func_ret_failure &&
-       (set x && func_ret_success y && test x = "[$]1")
+       (set x && func_ret_success y && test x = "[$]1") &&
+       subshell_umask_sanity
      '
      for gl_cv_posix_shell in \
         "$CONFIG_SHELL" "$SHELL" /bin/sh /bin/bash /bin/ksh /bin/sh5 no; do
--
1.6.0.2.229.g1293c