Re: [FYI] {maint} tests: report useful system information from 'test-suite.log'

[Stefano Lattarini]

Hi Eric.

On 12/22/2011 03:41 PM, Eric Blake wrote:
> On 12/22/2011 07:30 AM, Stefano Lattarini wrote:
>> * tests/get-sysconf.test: New test, gathering system information
>> and then always terminating with a SKIP, so that its output gets
>> copied in `test-suite.log'.
>> * tests/Makefile.am (TESTS): Add it.
> 
> I like the idea.  But I'm afraid it might introduce a few bug reports on
> its own ("I ran the testsuite, and it skipped 1 test - how do I fix
> things to not skip that test?").
>
Well, time will tell :-)  But it's worth noting that at least one test in
the automake testsuite is bound to be skipped anyway on almost every system:
some tests require libtool macros *in /usr/local/share/aclocal* (if one
has configured automake with the default prefix), some requires Vala, some
the UPC compiler, some the Sun java interpreter ... (you've got the gist I
guess), and nobody has ever complained so far about this situation.  So I
guess we're pretty safe.

> I also worry that since config.log can be long,
>
Luckily, the automake's one isn't (even in the branches that have started
looking for compilers at configure time).

> it might make the user's
> email long enough to trip up list posting limitations, and not all users
> know how to compress things to get around that.
>
If we come to that, we should raise the list's post size limitation IMO.

> And even if they do
> compress it, it's that much harder to read the testsuite failure report.
> 
> But I don't have any ideas for how to do better, so you might as well go
> with it.
>
Yeah, IMHO it's a definite improvement over the previous situation.  And
if we find a better solution in the future, well, we can simply implement
it, right? :-)

>> +# Dummy test case, aimed at reporting useful system information in the
>> +# final `test-suite.log'.  This way, if a user experiencing a failure in
>> +# the Automake test suite only sends us the `test-suite.log' file upon,
>> +# we won't have to ask him for more information to start analyzing the
>> +# failure (that had happened too many times in the past!).
> 
> grammar:
> 
> This way, if a user experiences a failure in the Automake test suite,
> then only sends us the `test-suite.log', we won't have to...
>
Will fix in a follow-up (probably this evening).

>> +
>> +if test $st -eq 0; then
>> +  # This test SKIPs, so that all the information is has gathered and
> 
> s/is has //
>
Ditto.

>> +  # printed will get unconditionally copied into the `test-suite.log'
>> +  # file.
>> +  Exit 77
>> +fi
>> +
>> +# Some unexpected error occurred; this must be reported as an hard error
>> +# by the testsuite driver.
>> +Exit 99
> 

Thanks,
  Stefano