Re: combined report for make check?

[Stefano Lattarini]

Hi Karl.

On 01/20/2013 11:20 PM, Karl Berry wrote:
> Especially for packages which run make check in several subdirectories
> (e.g., texinfo), Nelson Beebe made the suggestion that it would be
> helpful if there was an "overall" textual report of success or failure,
> and not just the exit status.
>
> Something like "ALL TESTS PASSED"
> vs. "FAILED TESTS".  Leaving it to the user to go back and examine the
> full log to see which ones in which directories.
>
This would require to change the 'check-recursive' targets not to
share the same code with the other '*-recursive' targets.  I really
don't want to go there.

The best solution is on the user-side IMHO: fix the build system to
use less (ideally none) make recursion.  Both the parallel and serial
testsuite harness should support that setup OOTB.

> The idea being that people (like Nelson) building on many systems could
> just grep for that string in the build logs and determine which ones
> need further inspection.
>
For that, a "screen scraping" script wrapping the 'make check' invocation
might the simplest and most backward-compatible solution (that is, it will
work also with all packages bootstrapped using "older" Automake releases).

If anyone wants to take a shot at such a script, and optimize it for use
with Automake-generated testsuite harnesses, I'd be very happy to carry
it in the 'contrib/' subdirectory of the Automake git repository.

> I'm not sure how feasible this is, but I thought I'd at least bring it up.
> 
Well done; doing so is always a good idea, whatever the final outcome is.

Thanks,
  Stefano