Re: Spurious tests are run in the old testsuite.

[Peter Rosin]

Den 2010-09-09 14:04 skrev Peter Rosin:
> Hi!
> 
> When I run this on a freshly bootstrapped, configured and made tree:
> 
> make check-TESTS TESTS="tests/mdemo-conf.test tests/mdemo-make.test 
> tests/mdemo-exec.test"
> 
> I get this output on Cygwin:
> 
> make[1]: Entering directory `/home/peda/libtool/git/cygwin'
> PASS: tests/mdemo-static.test
> PASS: tests/mdemo-static-make.test
> PASS: tests/mdemo-static-exec.test
> PASS: tests/mdemo-static-inst.test
> PASS: tests/mdemo-static-unst.test
> PASS: tests/mdemo-conf.test
> PASS: tests/mdemo-make.test
> PASS: tests/mdemo-exec.test
> ==================
> All 3 tests passed
> ==================
> make[1]: Leaving directory `/home/peda/libtool/git/cygwin'
> 
> So, it runs 8 tests but only counts the requested 3 three.

I hate to whine, but here I go anyway.  You should try to

$ mkdir newdir
$ cd newdir
$ .../configure
$ make
$ make check-interactive

and count how many extra tests that are run (37 extra tests, 51 instead
of 14).  Then multiply with the pain it is to run even a single test on
MSYS.  Then consider that a fair share of the bogus tests are tests that
run configure (8 extra configures, 10 instead of 2), and the wasted time
spent in those are of course ridiculous.

I can touch tests/demo-conf.log etc etc etc I suppose, but that is also
a pain.  And should not be necessary.

Now, I will not have any further use for check-interactive as I have
rigged my MSYS testsuite runs to (optionally) always be non-interactive
with the patch mentioned in another thread over at libtool-patches. So,
for the specific case of check-interactive this is not very important.
But it is a complete drag that you can't run only the test(s) you need.

Please remind me of the astounding performance benefits achieved
elsewhere with the parallelism in the testsuite, because I'm not seeing
any of that over here.

Cheers,
Peter