Re: automake 1.11.3 check-TESTS and command line length

[Stefano Lattarini]

On 02/22/2012 09:22 PM, Bob Friesenhahn wrote:
> On Wed, 22 Feb 2012, Stefano Lattarini wrote:
>>>
>> I don't understand how that patch could actually work ...  If there are
>> too many tests in $(TESTS), there will be too many logs in $(TEST_LOGS),
>> and since the recipe for $(TEST_SUITE_LOG) contents the expansion of
>> $(TEST_LOGS), the command line length limit for "/bin/sh -c" will be
>> exceeded anyway ...
> 
> I don't claim to understand how the patch functions other than
> that it seems to delegate responsibility to a subordinate make.
>
Basically, it was trying (and succeeding!) to pass the list of tests
to the subordinate make on the stdin rather than on the command line,
to avoid hitting any command-line length limit.  It was a pretty
simple and clever hack.  The problem is that the list of tests will
still *have* to be finally expanded in the recipe that creates
'test-suite.log', at which point all our attempt to avoid exceeding
command line length limits in recursive make invocations becomes
useless, no matter how clever of fast.

>>> A GNU make dependency would be a real problem.  A dependency on a
>>> properly-working standard make would not be such a problem.
>>>
>> Is the test suite actually failing due to "exceeded command line length"
>> on any system different from Cygwin or MinGW?  If not, I don't see how a
>> GNU make dependency would be a problem (if not for the fact that it
>> uglifies the already hideous patch even more).
> 
> I have only observed it on MinGW/MSYS but that does not mean it will
> not appear on some Unix OS with unusually short command line length.
>
True; but I say, let's cross that bridge when come at it :-)

> A GNU make dependency would be a problem if it depends on GNU make
> for other systems.
>
This wouldn't be the case: the change causes the GNU make specific code
to be used only if ./configure determines that the make program in use
is GNU make.  Otherwise, the old implementation would still been used.
So, no regression for the non-GNU makes (even if the pre-existing
command-line length isse wouldn't have been fixed for them).

> On MinGW, /bin/sh throws an error but I don't recall that it provides
> a useful diagnostic like "exceeded command line length".  It seemed
> to me that the argument list was truncated.
> 
>>> I have not run into any problems when using Ralf's patch, but it apparently
>>> does not address all issues so Ralf backed it out.
>>>
>> In fact, it fundamentally fails to address the issues -- that's why Ralf
>> backed it out.
> 
> All I know is that with the patch I did not experience a failure on any
> system.
>
That's weird; clearly I'm missing something in the big picture here ...
Maybe Peter can chime in with his MinGW expertise and save the day? ;-)

>> Are you referring to the GraphicsMagick testsuite?  If yes, it seems to me
>> that it could benefit greatly from the use of TAP once automake 1.12 is out;
>> for example, all the 314 'tests/rwfile*.sh' tests could be rewritten as one
>> single TAP test (and similarly for many other auto-generated tests), thus
>> removing the need for a huge number of files without reducing granularity
>> of the results.
> 
> Yes, this is for the GraphicsMagick test suite.  There is little doubt that
> the test suite is grossly inefficient.  The good news is that the test suite
> scripts are generated by a Makefile target (manually invoked) so it should
> be possible to output the tests in some other format.
>
Or even not generate them, but have some TAP tests doing the looping work
directly; for an example of a similar move from a bunch of autogenerated
tests to a single TAP test, you can take a look to this Automake commit:

 <http://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=8b4dfc2d>

Regards,
 Stefano