[PATCH] verify: automate tests

[Eric Blake]

Had we automated this sooner, we would have caught the issue
with gcc -Werror -Wredundant-decls sooner.

* modules/verify-tests: New module.
* tests/test-verify.sh: New file.
* tests/test-verify.c: Guard each negative test with a unique id.
Also avoid warning about unused left hand of comma expressions.

Signed-off-by: Eric Blake <address@hidden>
---

> What's worse, we have test-verify.c that was supposed to catch problems
> like this, but it is never getting run.  Why?
...
> Because there is no modules/verify-tests.  The other files above need to
> be evaluated - why do they exist if no test module mentions them?  But I
> will shortly be committing a patch to add the verify-tests module.

It turned out to be a tricky proposition to test expected compilation
failure, but I think the solution I came up with does quite well.

I'm pushing this.  I verified that if this patch is applied before
Paul's fixes to verify.h today, that
CFLAGS='-Werror -Wredundant-decls' ./gnulib-tool --with-tests --test verify
detects the failure; and after Paul's patch it is clean.
gnulib-tool --with-tests stressed a VPATH build.
I also tested an in-tree build by importing this into m4.git.

If there's fallout from this, we can address it in followup patches.

 ChangeLog            |    8 ++++++++
 modules/verify-tests |   13 +++++++++++++
 tests/test-verify.c  |   26 +++++++++++++++++++++++---
 tests/test-verify.sh |   20 ++++++++++++++++++++
 4 files changed, 64 insertions(+), 3 deletions(-)
 create mode 100644 modules/verify-tests
 create mode 100755 tests/test-verify.sh

diff --git a/ChangeLog b/ChangeLog
index 4897fb8..b92862f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,11 @@
+2010-05-05  Eric Blake  <address@hidden>
+
+       verify: automate tests
+       * modules/verify-tests: New module.
+       * tests/test-verify.sh: New file.
+       * tests/test-verify.c: Guard each negative test with a unique id.
+       Also avoid warning about unused left hand of comma expressions.
+
 2010-05-05  Paul Eggert  <address@hidden>

        Further improvements to verify.h, suggested by Eric Blake.
diff --git a/modules/verify-tests b/modules/verify-tests
new file mode 100644
index 0000000..936ed56
--- /dev/null
+++ b/modules/verify-tests
@@ -0,0 +1,13 @@
+Files:
+tests/test-verify.c
+tests/test-verify.sh
+tests/init.sh
+
+Depends-on:
+
+configure.ac:
+
+Makefile.am:
+TESTS_ENVIRONMENT += MAKE='$(MAKE)'
+TESTS += test-verify test-verify.sh
+check_PROGRAMS += test-verify
diff --git a/tests/test-verify.c b/tests/test-verify.c
index 890762f..47eae2b 100644
--- a/tests/test-verify.c
+++ b/tests/test-verify.c
@@ -21,12 +21,20 @@

 #include "verify.h"

+#ifndef EXP_FAIL
+# define EXP_FAIL 0
+#endif
+
 int x;
 enum { a, b, c };

+#if EXP_FAIL == 1
 verify (x >= 0);                  /* should give ERROR: non-constant 
expression */
+#endif
 verify (c == 2);                  /* should be ok */
+#if EXP_FAIL == 2
 verify (1 + 1 == 3);              /* should give ERROR */
+#endif
 verify (1 == 1); verify (1 == 1); /* should be ok */

 enum
@@ -36,13 +44,25 @@ enum

 int function (int n)
 {
+#if EXP_FAIL == 3
   verify (n >= 0);                  /* should give ERROR: non-constant 
expression */
+#endif
   verify (c == 2);                  /* should be ok */
+#if EXP_FAIL == 4
   verify (1 + 1 == 3);              /* should give ERROR */
+#endif
   verify (1 == 1); verify (1 == 1); /* should be ok */

   if (n)
-    return (verify_true (1 == 1), verify_true (1 == 1), 7); /* should be ok */
-  else
-    return (verify_true (1 == 2), 5); /* should give ERROR */
+    return ((void) verify_true (1 == 1), verify_true (1 == 1) + 7); /* should 
be ok */
+#if EXP_FAIL == 5
+  return (verify_true (1 == 2), 5); /* should give ERROR */
+#endif
+  return 0;
+}
+
+int
+main (void)
+{
+  return !(function (0) == 0 && function (1) == 8);
 }
diff --git a/tests/test-verify.sh b/tests/test-verify.sh
new file mode 100755
index 0000000..11b7311
--- /dev/null
+++ b/tests/test-verify.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+. "${srcdir=.}/init.sh"
+
+# Rather than figure out how to invoke the compiler with the right
+# include path ourselves, we let make do it:
+(cd "$initial_cwd_" && rm -f test-verify.o \
+    && $MAKE test-verify.o >/dev/null 2>&1) \
+  || skip_ "cannot compile error-free"
+
+# Now, prove that we encounter all expected compilation failures:
+: >out
+: >err
+for i in 1 2 3 4 5; do
+  (cd "$initial_cwd_"
+   rm -f test-verify.o
+   $MAKE CFLAGS=-DEXP_FAIL=$i test-verify.o) >>out 2>>err \
+  && { warn_ "compiler didn't detect verification failure $i"; fail=1; }
+done
+
+Exit $fail
-- 
1.6.6.1