RE: How do you set VERBOSE for parallel testin

[Arthur Schwarz]

> >>
> >> If any test causes a FAIL, XPASS, or ERROR condition, then 'make check'
> >> will likewise fail.  The output from make is telling you that your
> >> testsuite caught failures that should not be present if the testsuite
> >> were passing.
> >    
> >   My issue is that the output indicates that the rule for test-suite.log
> > fails. Is this expected? My thought (from the automake manual) is that
even
> > if there is a failure there should be no difficulty creating or
outputting
> > test-suite.log. The test harness created a test-suite.log (included as
an
> > attachment) but failed to output it. Don't know why.
> 
> Yes, this is expected.  test-suite.log is created no matter what, but
> the rules associated with creating it fail if the log contains any
> failure reports, so that make will quit running and let you investigate
> those failures.

If I understand this correctly, when executing test scripts the first
failure will cause make to discontinue execution (?). This doesn't happen.
Make continues until all tests in the variable TESTS are executed. Then
again, I must be misunderstanding you?

> 
> As for why test-suite.log was not replayed as part of the make output,
> I'm not sure that you had VERBOSE=1 set.  So far, you've only attached
> the generated Makefile (but not the source Makefile.am), and didn't show
> what command you ran.  You'll need to make your setup easier for others
> to reproduce before we can see what you are attempting.

I am including several files.
   test.pdf        console output showing test results
   Makefile.am     src/Makefile
   Test-suite.log  generated file
   configure.ac    ./configure.am
   Makefile.am     ./Makefile.am
> 
> As for the test-suite.log not having much output, it is probably because
> your tests are not very noisy.  Running make captured all of the output
> of your tests, but if your tests didn't output anything, then the log
> has very little to show.  Again, without knowing what your test scripts
> are doing, it's hard to reproduce your setup.

>From the Automake Manual Section 15.2.3 Parallel Test Harness, " The output
from failed tests is collected in the test-suite.log file. If the variable
'VERBOSE' is set, this file is output after the summary."

This quote is in fact not fully accurate. The output from all tests are
collected in the test-suite.log file. I can not set VERBOSE to output the
test-suite.log file.

Your statement concerning the paucity of output does not preclude showing
any output. In fact, removing the generated "echo" statements from the
generated shell scripts in Makefile.am, and making no other change, will
show that the test harness generates an output for each test case executed
in the test-suite.log file. 

I have tried to set VERBOSE in my Makefile.am to '1' and to 'yes' and "yes"
and just plain yes. The test-suite.log file does not output. Am I missing
something? How is VERBOSE supposed to be turned on?

Anyhow, I'm puzzled as to what to do and how to check this out. The general
comments probably apply to all variables which have a 'yes/no' value. What
should the value be to turn them on or off? A similar question occurs in
setting AM_COLOR_TEST to 'always'. What are the legitimate values?

Sorry about the delay in answering.
art
>

test.pdf
Description: Adobe PDF document

Makefile.am
Description: Binary data

test-suite.log
Description: Binary data

configure.ac
Description: Binary data

Makefile.am
Description: Binary data