[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

autotest: --keywords, and test titles

From: Ralf Wildenhues
Subject: autotest: --keywords, and test titles
Date: Tue, 7 Mar 2006 00:26:40 +0100
User-agent: Mutt/1.5.9i

I apologize for posting ever more patches in what was supposed to be
some stabilization work..

This patch and discussion
changed a comment in `--keyword' handling in lib/autotest/general.m4
| -    # It is on purpose that we match the test group titles too.

| +             # Do not match the test group titles.

But the new comment is not true: we do match (whole words) from test
group titles (AT_SETUP arguments): try 
  ./tests/testsuite --list -k and

for example.  The cited discussion is about adding back `test banners'
(AT_BANNER args) in the output, not about matching `test titles'.

Even before this change, the (unchanged) documentation did not match the
| `--keywords=KEYWORDS'
|      Add to the selection the test groups with title or keywords
|      (arguments to `AT_SETUP' or `AT_KEYWORDS') that match _all_
|      keywords of the comma separated list KEYWORDS.
|      Running `./testsuite -k autoupdate,FUNC' will select all the tests
|      tagged with `autoupdate' _and_ `FUNC' (as in `AC_CHECK_FUNC',
|      `AC_FUNC_FNMATCH', etc.) while `./testsuite -k autoupdate -k FUNC'
|      will select all tests tagged with `autoupdate' _or_ `FUNC'.

We use
  ...[; ]$at_keyword[; ]
to match; thus, to match the AT_SETUP([AC_FUNC_FNMATCH]) test,
  ./testsuite -k AC_FUNC_FNMATCH
would work, but the documented
  ./testsuite -k FUNC
would not.

A bit of history digging:

The code to also match titles was introduced first here:

And the change to match whole words only was done here:

Now, before I go and adjust the documentation and write some tests,
let's agree on desired semantics:
- should `--keywords' match test titles, as documented?
  (I think it should)
- should `--keywords' match anywhere, or only complete words?
  (I think complete words by default would be ok)

If we could agree on those, maybe we could also document
     -k '.*FUNC.*'
as a way to match parts of words, to get back the expressiveness
hinted at in the current documentation; arguably, that would limit
the implementation, but IMVHO it's more useful in the default case to
match whole words only: otherwise, getting it to actually match whole
words is quite involved.

Also, it'd be really cool to have keyword negation, so that the user can
do arbitrary logical operations on keywords (albeit in disjunctive
normal form only):
     ./testsuite -k 'libtoolize,!autoconf' -k 'libtool,!autoconf'

should test libtool and libtoolize on a system where Autoconf is not
installed (this is how CVS Libtool uses keywords at the moment).

Here's the code half of a patch.  Would `!' be a good character for
negation?  It is difficult to enter on the command line (single quotes
necessary in bash, for example).  What would be better?

If ok, I could do a documentation and testsuite update.  Thoughts?


        * lib/autotest/general.m4 (AT_INIT) <at_optarg>: Optimize
        `expr' away if there is nothing to do.
        < --keywords >: Simplify and robustify argument handling.
        Revert erroneous comment from 2005-08-23.  Extend to allow
        keywords negated with `!'.

Index: lib/autotest/general.m4
RCS file: /cvsroot/autoconf/autoconf/lib/autotest/general.m4,v
retrieving revision 1.197
diff -u -r1.197 general.m4
--- lib/autotest/general.m4     5 Mar 2006 08:29:50 -0000       1.197
+++ lib/autotest/general.m4     6 Mar 2006 17:10:02 -0000
@@ -273,7 +273,10 @@
-  at_optarg=`expr "x$at_option" : 'x[[^=]]*=\(.*\)'`
+  case $at_option in
+  *=*) at_optarg=`expr "x$at_option" : 'x[[^=]]*=\(.*\)'` ;;
+  *)   at_optarg= ;;
+  esac
   # Accept the important Cygnus configure options, so we can diagnose typos.
@@ -358,11 +361,23 @@
     --keywords=* )
-       for at_keyword in `IFS=,; set X $at_optarg; shift; echo address@hidden
+       at_save_IFS=$IFS
+       IFS=,
+       set X $at_optarg
+       shift
+       IFS=$at_save_IFS
+       for at_keyword
-               # Do not match the test group titles.
-               at_groups_selected=`echo "$at_groups_selected" |
-                       grep -i ["^[1-9][^;]*;.*[; ]$at_keyword[ ;]"]`
+         at_invert=
+         case $at_keyword in
+         '!'*)
+           at_invert="-v"
+           at_keyword=`expr "X$at_keyword" : '..\(.*\)'`
+           ;;
+         esac
+         # It is on purpose that we match the test group titles too.
+         at_groups_selected=`echo "$at_groups_selected" |
+             grep -i $at_invert ["^[1-9][^;]*;.*[; ]$at_keyword[ ;]"]`
        at_groups_selected=`echo "$at_groups_selected" | sed 's/;.*//'`
        # Smash the newlines.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]