[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[FYI] {testsuite-work} maintcheck: consistency of list of test scripts
From: |
Stefano Lattarini |
Subject: |
[FYI] {testsuite-work} maintcheck: consistency of list of test scripts |
Date: |
Tue, 3 May 2011 19:11:07 +0200 |
User-agent: |
KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) |
Reference:
<http://lists.gnu.org/archive/html/automake-patches/2011-04/msg00141.html>
On Tuesday 19 April 2011, Stefano Lattarini wrote:
> Wow, this is an oooold one. References:
> <http://lists.gnu.org/archive/html/automake-patches/2009-12/msg00026.html>
> <http://lists.gnu.org/archive/html/automake-patches/2009-12/msg00045.html>
>
> -*-*-
>
> maintcheck: consistency of list of test scripts
>
> * CheckListOfTests.am: New file.
> (maintainer-check-list-of-tests): New target,
> check for consistency between list of tests defined in the
> including Makefile and list of tests on the filesystem.
> (clean-maintcheck-testslist-tmp): New rule, to clean up the
> temporary files that might be left around by the rules associated
> with the previous target.
> (clean-local): Depend on it.
> * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
> (maintainer-check): Added dependency from the new target
> `maintainer-check-list-of-tests'.
> * tests/Makefile.am: Likewise. Also ...
> (TEST_EXTENSIONS): New, define to `.test'.
> ---
> ChangeLog | 21 ++++++++++++
> CheckListOfTests.am | 70
> ++++++++++++++++++++++++++++++++++++++++
> Makefile.am | 21 +++++++++++-
> Makefile.in | 18 ++++++++++-
> lib/Automake/tests/Makefile.am | 5 ++-
> lib/Automake/tests/Makefile.in | 68 ++++++++++++++++++++++++++++++++------
> tests/Makefile.am | 5 +++
> tests/Makefile.in | 52 ++++++++++++++++++++++++++++--
> 8 files changed, 243 insertions(+), 17 deletions(-)
> create mode 100644 CheckListOfTests.am
>
> -*-*-
>
> OK for maint? I'll push in 72 hours if there is no objection.
>
> Regards,
> Stefano
>
OK, I had to revise this patch a bit now that in the 'testsuite-work'
branch we have stuff in $(TESTS) that doesn't correspond to existing
files (by a conscious choice). The updated patch, still based of off
maint, is attached.
I've merged in into 'testsuite-work' and pushed. I hope it will be
accepted also in maint some not-too-distant day.
Regards,
Stefano
From 7b6ab07442b28a62389fc4dc8f35276f31348c13 Mon Sep 17 00:00:00 2001
Message-Id: <address@hidden>
From: Stefano Lattarini <address@hidden>
Date: Sat, 30 Jan 2010 01:23:44 +0100
Subject: [PATCH] maintcheck: consistency of list of test scripts
* CheckListOfTests.am: New file.
(maintainer-check-list-of-tests): New target,
check for consistency between list of tests defined in the
including Makefile and list of tests on the filesystem.
(clean-maintcheck-testslist-tmp): New rule, to clean up the
temporary files that might be left around by the rules associated
with the previous target.
(clean-local): Depend on it.
* lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
(maintainer-check): Added dependency from the new target
`maintainer-check-list-of-tests'.
* tests/Makefile.am: Likewise.
* Makefile.am (maintainer-check-list-of-test): New target,
calling recursively into `tests/' and `lib/Automake/tests/',
using ...
(TEST_SUBDIRS): ... this new variable.
---
ChangeLog | 20 ++++++++++
CheckListOfTests.am | 76 ++++++++++++++++++++++++++++++++++++++++
Makefile.am | 21 ++++++++++-
Makefile.in | 18 +++++++++-
lib/Automake/tests/Makefile.am | 5 ++-
lib/Automake/tests/Makefile.in | 70 +++++++++++++++++++++++++++++++------
tests/Makefile.am | 3 ++
tests/Makefile.in | 52 ++++++++++++++++++++++++++-
8 files changed, 249 insertions(+), 16 deletions(-)
create mode 100644 CheckListOfTests.am
diff --git a/ChangeLog b/ChangeLog
index 99e4f11..846b569 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,23 @@
+2011-05-03 Stefano Lattarini <address@hidden>
+
+ maintcheck: consistency of list of test scripts
+ * CheckListOfTests.am: New file.
+ (maintainer-check-list-of-tests): New target,
+ check for consistency between list of tests defined in the
+ including Makefile and list of tests on the filesystem.
+ (clean-maintcheck-testslist-tmp): New rule, to clean up the
+ temporary files that might be left around by the rules associated
+ with the previous target.
+ (clean-local): Depend on it.
+ * lib/Automake/tests/Makefile.am: Include `CheckListOfTests.am'.
+ (maintainer-check): Added dependency from the new target
+ `maintainer-check-list-of-tests'.
+ * tests/Makefile.am: Likewise.
+ * Makefile.am (maintainer-check-list-of-test): New target,
+ calling recursively into `tests/' and `lib/Automake/tests/',
+ using ...
+ (TEST_SUBDIRS): ... this new variable.
+
2011-04-24 Stefano Lattarini <address@hidden>
tests: fix spurious failure in a test on TESTS (VPATH-related)
diff --git a/CheckListOfTests.am b/CheckListOfTests.am
new file mode 100644
index 0000000..28cb795
--- /dev/null
+++ b/CheckListOfTests.am
@@ -0,0 +1,76 @@
+## -*- Automake -*-
+##
+## Copyright (C) 2011 Free Software Foundation, Inc.
+##
+## This program is free software; you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation; either version 2, or (at your option)
+## any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program. If not, see <http://www.gnu.org/licenses/>.
+##
+
+## Temporary files used in the `check-list-of-tests' target.
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
+
+## Check that the list of tests given in the Makefile is equal to the
+## list of all test scripts in the Automake testsuite.
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+## Prefer unified diffs over plain diffs, for readability.
+ @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+ diff='diff -u'; \
+ else \
+ diff='diff'; \
+ fi; \
+## Determine which files are to be considered test scripts.
+## Do not blindly look for `$(TEST_EXTENSIONS)', because the parallel
+## testsuite driver provided by Automake allow entries in `$(TESTS)'
+## not to be real files, but rather paramaters for the corresponding
+## LOG_COMPILER (well, sort of).
+ set X $(checked_test_extensions); shift; \
+ if test $$# -eq 0; then \
+ set X $(TEST_EXTENSIONS); shift; \
+ fi; \
+ if test $$# -eq 0; then \
+ echo '$@: internal error: $$(checked_test_extensions) and' \
+ '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+ exit 255; \
+ fi; \
+## List of tests in Makefile.
+ lst='$(expected_list_of_tests)'; \
+ test -n "$$lst" || lst='$(TESTS)'; \
+ for t in $$lst; do \
+ echo "$$t"; \
+ done | sort >$(am__tmk); \
+## List of tests on filesystem. Be careful to cater for VPATH builds too.
+ for ext in $$*; do \
+ ls *$$ext 2>/dev/null; \
+ if test $(srcdir) != $(builddir); then \
+ (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+ fi; \
+ done | sort | uniq >$(am__tfs); \
+## Compare the two lists, complain if they differ.
+ if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+ result=0; \
+ else \
+ echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+ echo "+ $$diff in-makefile on-filesystem" >&2; \
+ cat $(am__tdf) >&2; \
+ result=1; \
+ fi; \
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+ exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf)
diff --git a/Makefile.am b/Makefile.am
index 6e8b7df..aeff613 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -24,6 +24,8 @@
## run aclocal and automake.
SUBDIRS = lib . doc m4 tests
+TEST_SUBDIRS = tests lib/Automake/tests
+
bin_SCRIPTS = automake aclocal
CLEANFILES = $(bin_SCRIPTS)
@@ -150,7 +152,24 @@ sc_at_in_texi
.PHONY: $(syntax_check_rules)
$(syntax_check_rules): automake aclocal
-maintainer-check: $(syntax_check_rules)
+
+## Check that the list of tests given in the Makefile is equal to the
+## list of all test scripts in the Automake testsuite.
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @failcom='exit 1'; \
+ for f in x $$MAKEFLAGS; do \
+ case $$f in \
+ *=* | --[!k]*);; \
+ *k*) failcom='fail=yes';; \
+ esac; \
+ done; \
+ for subdir in $(TEST_SUBDIRS); do \
+ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \
+ done; \
+ test -z "$$fail"
+
+maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests
## Look for test whose names can cause spurious failures when used as
## first argument to AC_INIT (chiefly because they might contain an
diff --git a/Makefile.in b/Makefile.in
index 49c35f5..ea63d6b 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -236,6 +236,7 @@ top_build_prefix = @top_build_prefix@
top_builddir = @top_builddir@
top_srcdir = @top_srcdir@
SUBDIRS = lib . doc m4 tests
+TEST_SUBDIRS = tests lib/Automake/tests
bin_SCRIPTS = automake aclocal
CLEANFILES = $(bin_SCRIPTS)
AUTOMAKESOURCES = automake.in aclocal.in
@@ -928,7 +929,22 @@ dist-hook:
.PHONY: $(syntax_check_rules)
$(syntax_check_rules): automake aclocal
-maintainer-check: $(syntax_check_rules)
+
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @failcom='exit 1'; \
+ for f in x $$MAKEFLAGS; do \
+ case $$f in \
+ *=* | --[!k]*);; \
+ *k*) failcom='fail=yes';; \
+ esac; \
+ done; \
+ for subdir in $(TEST_SUBDIRS); do \
+ (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) $@) || eval $$failcom; \
+ done; \
+ test -z "$$fail"
+
+maintainer-check: $(syntax_check_rules) maintainer-check-list-of-tests
sc_test_names:
@m4_builtin_rx=`echo $(m4_builtins) | sed 's/ /|/g'`; \
m4_macro_rx="\\<($$m4_builtin_rx)\\>|\\<_?(A[CUMHS]|m4)_"; \
diff --git a/lib/Automake/tests/Makefile.am b/lib/Automake/tests/Makefile.am
index c5e53d2..fad3d7e 100644
--- a/lib/Automake/tests/Makefile.am
+++ b/lib/Automake/tests/Makefile.am
@@ -1,6 +1,7 @@
## Process this file with automake to create Makefile.in
-# Copyright (C) 2002, 2003, 2008, 2009 Free Software Foundation, Inc.
+# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation,
+# Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -28,3 +29,5 @@ Version.pl \
Wrap.pl
EXTRA_DIST = $(TESTS)
+
+include $(top_srcdir)/CheckListOfTests.am
diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index b4940db..bdf7219 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -15,7 +15,8 @@
@SET_MAKE@
-# Copyright (C) 2002, 2003, 2008, 2009 Free Software Foundation, Inc.
+# Copyright (C) 2002, 2003, 2008, 2009, 2010 Free Software Foundation,
+# Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -47,8 +48,9 @@ NORMAL_UNINSTALL = :
PRE_UNINSTALL = :
POST_UNINSTALL = :
build_triplet = @build@
+DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
+ $(top_srcdir)/CheckListOfTests.am
subdir = lib/Automake/tests
-DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \
$(top_srcdir)/m4/auxdir.m4 $(top_srcdir)/m4/init.m4 \
@@ -279,11 +281,14 @@ Version.pl \
Wrap.pl
EXTRA_DIST = $(TESTS)
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
all: all-am
.SUFFIXES:
.SUFFIXES: .html .log .pl
-$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps)
+$(srcdir)/Makefile.in: $(srcdir)/Makefile.am
$(top_srcdir)/CheckListOfTests.am $(am__configure_deps)
@for dep in $?; do \
case '$(am__configure_deps)' in \
*$$dep*) \
@@ -518,7 +523,7 @@ maintainer-clean-generic:
@echo "it deletes files that may require special tools to rebuild."
clean: clean-am
-clean-am: clean-generic mostlyclean-am
+clean-am: clean-generic clean-local mostlyclean-am
distclean: distclean-am
-rm -f Makefile
@@ -586,17 +591,60 @@ uninstall-am:
recheck-html
.PHONY: all all-am check check-TESTS check-am check-html clean \
- clean-generic distclean distclean-generic distdir dvi dvi-am \
- html html-am info info-am install install-am install-data \
- install-data-am install-dvi install-dvi-am install-exec \
- install-exec-am install-html install-html-am install-info \
- install-info-am install-man install-pdf install-pdf-am \
- install-ps install-ps-am install-strip installcheck \
- installcheck-am installdirs maintainer-clean \
+ clean-generic clean-local distclean distclean-generic distdir \
+ dvi dvi-am html html-am info info-am install install-am \
+ install-data install-data-am install-dvi install-dvi-am \
+ install-exec install-exec-am install-html install-html-am \
+ install-info install-info-am install-man install-pdf \
+ install-pdf-am install-ps install-ps-am install-strip \
+ installcheck installcheck-am installdirs maintainer-clean \
maintainer-clean-generic mostlyclean mostlyclean-generic pdf \
pdf-am ps ps-am recheck recheck-html uninstall uninstall-am
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+ diff='diff -u'; \
+ else \
+ diff='diff'; \
+ fi; \
+ set X $(checked_test_extensions); shift; \
+ if test $$# -eq 0; then \
+ set X $(TEST_EXTENSIONS); shift; \
+ fi; \
+ if test $$# -eq 0; then \
+ echo '$@: internal error: $$(checked_test_extensions) and' \
+ '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+ exit 255; \
+ fi; \
+ lst='$(expected_list_of_tests)'; \
+ test -n "$$lst" || lst='$(TESTS)'; \
+ for t in $$lst; do \
+ echo "$$t"; \
+ done | sort >$(am__tmk); \
+ for ext in $$*; do \
+ ls *$$ext 2>/dev/null; \
+ if test $(srcdir) != $(builddir); then \
+ (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+ fi; \
+ done | sort | uniq >$(am__tfs); \
+ if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+ result=0; \
+ else \
+ echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+ echo "+ $$diff in-makefile on-filesystem" >&2; \
+ cat $(am__tdf) >&2; \
+ result=1; \
+ fi; \
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+ exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf)
+
# Tell versions [3.59,3.63) of GNU make to not export all variables.
# Otherwise a system limit (for SysV at least) may be exceeded.
.NOEXPORT:
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 49d8942..e6d3098 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -847,6 +847,9 @@ $(parallel_tests)
EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS)
+include $(top_srcdir)/CheckListOfTests.am
+maintainer-check-list-of-tests: $(parallel_tests)
+
# Each test case depends on defs, aclocal, and automake.
check_SCRIPTS = defs aclocal-$(APIVERSION) automake-$(APIVERSION)
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 78d723c..76800aa 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -50,7 +50,7 @@ POST_UNINSTALL = :
build_triplet = @build@
DIST_COMMON = README $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
$(srcdir)/aclocal.in $(srcdir)/automake.in $(srcdir)/defs.in \
- $(srcdir)/parallel-tests.am
+ $(srcdir)/parallel-tests.am $(top_srcdir)/CheckListOfTests.am
subdir = tests
ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \
@@ -1117,6 +1117,9 @@ yflags2.test \
$(parallel_tests)
EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS)
+am__tmk = tests-in-makefile-list.tmp
+am__tfs = tests-on-filesystem-list.tmp
+am__tdf = diff-in-tests-lists.tmp
# Each test case depends on defs, aclocal, and automake.
check_SCRIPTS = defs aclocal-$(APIVERSION) automake-$(APIVERSION)
@@ -1124,7 +1127,7 @@ all: all-am
.SUFFIXES:
.SUFFIXES: .html .log .test
-$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am
$(am__configure_deps)
+$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am
$(top_srcdir)/CheckListOfTests.am $(am__configure_deps)
@for dep in $?; do \
case '$(am__configure_deps)' in \
*$$dep*) \
@@ -1455,6 +1458,51 @@ $(parallel_tests): $(parallel_tests:-p.test=.test)
Makefile.am
< $(srcdir)/$$input >$@
chmod a+rx $@
+.PHONY: maintainer-check-list-of-tests
+maintainer-check-list-of-tests:
+ @if diff -u /dev/null /dev/null >/dev/null 2>&1; then \
+ diff='diff -u'; \
+ else \
+ diff='diff'; \
+ fi; \
+ set X $(checked_test_extensions); shift; \
+ if test $$# -eq 0; then \
+ set X $(TEST_EXTENSIONS); shift; \
+ fi; \
+ if test $$# -eq 0; then \
+ echo '$@: internal error: $$(checked_test_extensions) and' \
+ '$$(TEST_EXTENSIONS) are both undefined or empty' >&2; \
+ exit 255; \
+ fi; \
+ lst='$(expected_list_of_tests)'; \
+ test -n "$$lst" || lst='$(TESTS)'; \
+ for t in $$lst; do \
+ echo "$$t"; \
+ done | sort >$(am__tmk); \
+ for ext in $$*; do \
+ ls *$$ext 2>/dev/null; \
+ if test $(srcdir) != $(builddir); then \
+ (cd $(srcdir) && ls *$$ext 2>/dev/null); \
+ fi; \
+ done | sort | uniq >$(am__tfs); \
+ if $$diff $(am__tmk) $(am__tfs) >$(am__tdf); then \
+ result=0; \
+ else \
+ echo '$@: list of tests in Makefile an on filesystem differ' >&2; \
+ echo "+ $$diff in-makefile on-filesystem" >&2; \
+ cat $(am__tdf) >&2; \
+ result=1; \
+ fi; \
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf); \
+ exit $$result;
+
+.PHONY: clean-maintcheck-testslist-tmp
+clean-local: clean-maintcheck-testslist-tmp
+clean-maintcheck-testslist-tmp:
+ rm -f $(am__tmk) $(am__tfs) $(am__tdf)
+
+maintainer-check-list-of-tests: $(parallel_tests)
+
clean-local: clean-local-check
.PHONY: clean-local-check
clean-local-check:
--
1.7.2.3
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [FYI] {testsuite-work} maintcheck: consistency of list of test scripts,
Stefano Lattarini <=