Re: testsuite failure - 193 parallel execution

[Eric Blake]

On 07/20/2010 01:27 PM, Ralf Wildenhues wrote:
> Hello Paul, Eric,
> 
> * Paul Eggert wrote on Tue, Jul 20, 2010 at 08:39:45PM CEST:
>> System call writes and reads should be atomic, but there's no requirement
>> that the shell must implement "echo token" as an atomic write of 6 bytes.
>> If the shell sometimes implements "echo token" as multiple writes, that alone
>> would explain the problem.
> 
> Good point, but wouldn't that be at least a QoI issue for the shell?

Bash is already doing a single write():

$ strace sh -c 'echo token > /dev/null' 2>&1 | grep write
write(1, "token\n", 6)                  = 6

> 
>> I suggest replacing "echo token" with "echo ''", so that a token
>> is always a single newline byte.  This may have nothing to do with
>> your problem, but it should be done anyway, to make the script
>> more portable.  And if bash or the kernel does have bugs with respect to
>> atomicity, it will make the script less liable to run into the bugs.
> 
> OK; that may still be more than one byte on w32 systems, but it's an
> improvement nonetheless.

I agree that it won't help shells that already do a single write(), but
at the same time, that it is worth applying.

Hopefully, w32 systems shouldn't be a problem.  To my knowledge, no one
but cygwin provides named fifos on the command line, and cygwin's named
fifos are always opened in binary mode (only disk files have the
text/binary mode issue).  Well, that is, if cygwin named fifos can ever
be made to work (as of cygwin 1.7.5, there are still bugs in the cygwin
implementation that prevent parallel autotest, and this is already
documented in autoconf's BUGS file).

> 
> Eric, does this improve tests on your system?

Unfortunately, I'm still seeing failures:

$ for i in $(seq 100); do sh ./micro-suite -j4 > stdout || { echo fail
on $i; break; }; grep 'All 16' stdout || { echo fail2 on $i; break; }; done
...
./micro-suite: line 1726: echo: write error: Broken pipe
./micro-suite: line 4: echo: write error: Broken pipe
./micro-suite: line 1726: echo: write error: Broken pipe
./micro-suite: line 4: echo: write error: Broken pipe
./micro-suite: line 1726: echo: write error: Broken pipe
./micro-suite: line 4: echo: write error: Broken pipe
./micro-suite: line 1726: echo: write error: Broken pipe
./micro-suite: line 4: echo: write error: Broken pipe
fail2 on 30
$ grep All stdout
All 12 tests were successful.

So I still got short-changed, although this time it took more
iterations.  The only other thing that I can think of is that I know
bash has had reported bugs in the past where built-ins that produce
output (and particularly built-ins that don't end output in a newline)
don't always flush correctly across redirections; maybe putting the echo
>&6 in a subshell, or using /bin/echo to force an exec(), will force
bash to be cleaner?  Trying those ideas now...

-- 
Eric Blake   address@hidden    +1-801-349-2682
Libvirt virtualization library http://libvirt.org

signature.asc
Description: OpenPGP digital signature