Re: bug#8846: coreutils-8.12 on HP-UX 11.31: 3 of 365 tests failed

[Stefano Lattarini]

On Monday 13 June 2011, Eric Blake wrote:
> On 06/13/2011 02:32 PM, Stefano Lattarini wrote:
> >> +++ b/tests/init.sh
> >> @@ -317,6 +317,11 @@ path_prepend_ ()
> >>
> >>  setup_ ()
> >>  {
> >> +  # If we're redirecting a file descriptor larger than 2, say via 
> >> automake's
> >> +  # TESTS_ENVIRONMENT, that redirected FD is closed-on-exec on some 
> >> systems
> >> +  # (at least Solaris 10 and HP-UX 11.x), so redirect it here again.
> >> +  test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"
> >> +
> >>
> > ... isn't this equivalent to just using ">&2" unconditionally in 'warn_()'?
> 
> Not quite, in coreutils' current usage.
> 
> Coreutils sets stderr_fileno_=9 in init.cfg, not TESTS_ENVIRONMENT;
> which means that stderr can be changed after a call to setup_ but before
> a call to warn_.
>
True.  I expressed myself badly; my point is, now coreutils will behave
as if `$stderr_fileno_' wouldn't have been set in `init.cfg' -- which I
reckon is hardly what we want.

> Thus, we want to output to $stderr_fileno_ (which is 9, and is always the
> original stderr), even during a test where stderr has been redirected
> somewhere else.
>
Then I'm missing something crucial here; my mental picture of what's going
on here is the following:

 [1] TESTS_ENVIRONMENT redirects fd 9 to fd 2, which is the original stderr
     of the make process (usually a user tty);

 [2] The automake parallel-tests driver redirect fds 1 and 2 to the .log
     file associated with the test; but fd 9 still points to the orginal
     stderr, not to the log file (which is what we want);

 [3] The $(SHELL), which here we assume is ksh with close-on-exec, does a
     fork+exec to execute the test script; thus, when that script begins
     its execution, it has the fd 9 closed (d'oh!, but expected now);

 [4] the scripts sources init.sh, which in turn:
   [4.1] sources init.cfg, which defines $stderr_fileno_ to 9 (which,
         mind you, is a closed fd at this point);
   [4.2] executes setup_, which sees that $stderr_fileno_ is *not* set to
         `2', and thus re-define it to `2', thus directing it to the log
         file.

The net outcome is that the original stderr is lost, and that, by default,
warn_ and company will output their messages into the log file.  If this
is the effect we want, we can obtain it much more simply by removing the
`exec 9>&2' redirection from TESTS_ENVIRONMENT, and removing the code
snippet `test $stderr_fileno_ = 2 || eval "exec $stderr_fileno_>&2"' from
tests/init.sh.

> > 
> > IMHO, the right fix is to to modify the code in TESTS_ENVIRONMENT to avoid 
> > the
> > definition of $stderr_fileno_ the shell performs closed-on-exec; e.g.,
> > 
> >   TESTS_ENVIRONMENT = ...; \
> >     if test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = 
> > x'foo'; then
> >        stderr_fileno_=9; export stderr_fileno_;
> >     else
> >        unset stderr_fileno_ || :
> >     fi
> 
> But setting things during TESTS_ENVIRONMENT does seem a bit more robust.
>  Or rather, I think that the best action is to set stderr_fileno_
> alongside the code that does the redirection; if it is TESTS_ENVIRONMENT
> that does the exec 9>&2, then TESTS_ENVIRONMENT does seem like the saner
> place (rather than init.cfg) for setting stderr_fileno_.
>
I fully agree with this (even if it won't help in our particualr situation
IMHO).

> > 
> > If we know that bash and zsh are well behaved, we can even avoid a couple of
> > forks (Cygwin users won't hate us too much then):
> > 
> >  TESTS_ENVIRONMENT = ...; \
> >     if test -n "$${ZSH_VERSION}$${BASH_VERSION}" || \
> >        test x"`(exec 9>&1 && sh -c 'echo foo >&9' >/dev/null 2>&1)`" = 
> > x'foo'
> >     then
> >        stderr_fileno_=9; export stderr_fileno_;
> >     else
> >        unset stderr_fileno_ || :
> >     fi
> > 
> > A better fix would be to do the redirect $stderr_fileno_>&2 in tess/init.sh
> > iff $stderr_fileno_ is closed, but how can that be portably determined
> > without printing trash on the user screen (and for *each* test)?
> 
> Not possible to portably sniff out closed fds; quoting the autoconf manual:
> 
> > Don't rely on duplicating a closed file descriptor to cause an
> > error.  With Solaris @command{/bin/sh}, when the redirection fails, the
> > output goes to the original file descriptor.
> 
> 
I suspected that; see my later mail:
 <http://lists.gnu.org/archive/html/bug-coreutils/2011-06/msg00074.html>
for an alternative, simpler proposal.

Regards,
  Stefano