[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Automake testsuite misuses DejaGnu

From: Jacob Bachmeyer
Subject: Automake testsuite misuses DejaGnu
Date: Sun, 11 Jul 2021 23:02:48 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20090807 MultiZilla/ SeaMonkey/1.1.17 Mnenhy/

I was planning to find a solution with a complete patch before mentioning this, but since a release is imminent I will just state the problem: several tests in the Automake testsuite misuse DejaGnu and fail with the 1.6.3 DejaGnu release as a result.

DejaGnu has always required a DejaGnu testsuite to be rooted at a "testsuite" directory and this has long been documented in the manual. However, prior to 1.6.3, DejaGnu did not actually depend on this requirement being met. Changes during the development process to properly support non-recursive Automake makefiles required relying on this requirement to resolve the ambiguity between recursive and non-recursive usage. Several tests in the Automake testsuite do not meet this requirement and fail if run with DejaGnu 1.6.3.

The simple change of updating the tests to use a testsuite/ directory causes the tests to fail with older versions of DejaGnu, due to lack of support for non-recursive "make check" in those versions. I have not yet tried a patch that also switches the tests to use recursive make, but I believe that is probably the only way for the tests to pass with old and new DejaGnu.

Note that, according to the original author, Rob Savoye, DejaGnu has always been intended to require that testsuites be rooted at a "testsuite" directory and the behavior that Automake's test cases rely on was never supported.

The affected tests are: check12, dejagnu3, dejagnu4, dejagnu5, dejagnu6, dejagnu7, dejagnu-absolute-builddir, dejagnu-relative-srcdir, dejgnu-siteexp-extend, dejagnu-siteexp-useredit.

Note that these tests do not all fail with the 1.6.3 release, but will all fail with some future release when the undocumented support for a testsuite not rooted at "testsuite" will eventually be removed.

-- Jacob

reply via email to

[Prev in Thread] Current Thread [Next in Thread]