[PATCH v2] Overhauled and modularized tests in `instspc.test'.

[Stefano Lattarini]

Hi Ralf, thanks for the helpful review.

Your objections, reasonings and suggestsions have been quite enlighting,
and brought me to the understandanding that most of our problems were
deriving by the juggling of "weird strings" ;-> between gen-instspc-tests,
instspc.sh and the generated Makefile.in.  Thus I've attempted a rewrite
of the patch, where names and contents of the problematic strings are kept
in a single, centralized place: a new "driver script" `instspc-tests.sh'.

The updated patch is attached.  Let me now what you think of it.

Regards,
   Stefano

-*-*-

For completeness, I've adressed some of your objections and observations
below anyway, notwithstanding the complete patch rewrite.

On Wednesday 15 September 2010, Ralf Wildenhues wrote:
> * Stefano Lattarini wrote on Tue, Sep 14, 2010 at 02:15:59PM CEST:
> > On Monday 13 September 2010, Ralf Wildenhues wrote:
> > > The patch introduces significant extra testsuite overhead.
> > 
> > How much, quantitatively speaking?
> 
> Dunno.  Well, it adds 2*N aclocal, autoconf, and automake
> invocations with N = 45, IIUC.  That's something.  On the plus
> side, it allows more parallelism.
Yes, my assessment might be skewed by the fact that I alway run a
lot of tests in parallel (and in lower priority to avoid freezing
my desktop).

> We need to think about line length issues for our testsuite,
> though, due to long $(TESTS).  I will followup separately about
> this topic.
Thanks.

> > While writing the patch, I was
> > focused on correctness (which has been tricky enough to get
> > right, BTW), so I haven't noticed performance problems (since
> > they've not been dramatic IMHO).
> 
> Have you done testing on w32 (MinGW/MSYS or Cygwin) before,
> where typical testsuite runs already take a day or so?
Nope, no Windows system here, and no plan to install it either.
Sorry.

> The question is how much incremental gain this patch brings.
Let's judge the new version ;-)
 
> > > Would it be possible to unify build and install tests pairwise?
> > 
> > Not easily, since the "weird chars" causing expected failures for
> > install tests are different from those causing expected failures
> > for build tests.
> 
> Hmpf.
> 
> > > > --- /dev/null
> > > > +++ b/tests/gen-instspc-tests
> > > > 

> > > "weird" is colloquial, and also not very fitting IMVHO.  Please
> > > use just $file or so, like the original code did.
> > 
> > But they are not file names, they are... well, if not weird, at
> > least "problematic" chars.  The later usages in instspc.sh
> > confirm this IMO. What about using "problematic_chars" or
> > "problematic_string" instead?
> 
> That is so long.  "file" or "name" is so nice and short, yet more
> descriptive than "b" and "d" which I inferred you used for
> directory and base part of a file name ... ;-)
This problem issued should have mostly disappeared in the new version;
let me know if you still have objections regarding the variable names.

> >  ## The ":;" works around a bug in Bash 3.2 and earlier.  See
> >  section ## "Limitations of Shell Builtins" in the Autoconf
> >  manual for more info.
> > 
> > OK?
> 
> Here's a general rule: in general, all portability issues relative
> to Posix are documented in the Autoconf manual; only exceptions
> need special mention.  Can you find a good place to stick that
> rule, either in some piece of Automake documentation, or on a note
> near your screen, so that we don't have to keep repeating it in
> the code?  That way, you can shorten this to:
> 
>   ## The ":;" works around a bug in Bash 3.2 and earlier.
> 
> Thanks.
I'll try harder to abide by this rule, even if I still think that
adding a reference to the proper section of the Autoconf manual might
be vauable sometimes.  Not a big deal though.

> > > > +  echo '#!/bin/sh'; \
> > > > +  echo '# DO NOT EDIT! GENERATED AUTOMATICALLY!'; \
> > > > +  echo 'required=gcc # FIXME: any C compiler should be ok!'; \
> > > 
> > > I don't understand the FIXME.  Either any compiler is ok, in
> > > which case we could remove the line, or it isn't, in which
> > > case there is nothing to fix.
BTW, I dropped the `required' in the new version, as this is an
orthoghonal issue anyway, and not relevant for the present patch.

> > Well, we currently don't have a way to require a generic C
> > compiler in out tests (patches pending).  The "FIXME" comment
> > helps us to remember that once this limitation is lifted, the
> > script *might* be adjusted accordingly.
> 
> OK, here's a hint: 50% of all Automake tests require a C compiler,
> without being annotated; and as far as I can see, the pending patch
> series does not really fix that.
Yes it does.
> And it doesn't need to, either:
> AC_PROG_CC will cause configure to exit 77 if no C compiler is
> found. You shouldn't need to annotate required=cc.
> 
> Sorry for not having thought of this earlier, would've saved you
> some work in the other patch series I guess.
I still think that requiring simple cc and c++ (etc...) has merits.
But this is for another thread.
 
> > > > +         echo '. ./defs || Exit 99 || exit 99'; \
> > > 
> > > ;-)
> > 
> > A similar tweaking (for all tests) is planned in my ongoing
> > "Testsuite initialization refactoring".  Do you want me to drop
> > it in this patch and re-insert it when the refactoring branch is
> > merged?
> 
> No that's fine, I was just smiling at this.  I don't think we need
> the extra '|| exit 99' though: although it is more correct,
Even if I must admit that still have to see a shell that doesn't
abort by itself on a failed ". ./FILE" command... ;-)
> and going from exit to Exit introduced an ever so tiny regression
> there for when some user intentionally removed defs or made it
> unreadable, it *really* looks like over-engineering to me.
> 
> > What about these two comments instead?
> > 
> >  ---
> >  
> >   # Some of the above "problematic" characters cannot be used in
> >   the # name of a build directory on a POSIX host.  This list
> >   should be # empty, but is not due to limitations in Autoconf,
> >   Automake, Make, # M4, or the shell.
> >  
> >  ---
> >  
> >   # Similarly, some of the above "problematic" characters cannot
> >   be # used in the name of an install directory on a POSIX host.
> >    Ideally, # this list also should be empty.
> >  
> >  ---
> 
> Sure.
Merged into a single one in the new version.

> > >  <http://producingoss.com/en/producingoss.html#territoriality>
> > 
> > "In any case, credit information can already be obtained from the
> > 
> >  version control logs and other out-of-band mechanisms like
> >  mailing list archives, so no information is lost by banning it
> >  from the source files themselves".
> > 
> > ... I agree with this.  What about getting rid of all "author
> > tags" in the Automake tree?  This should obviously be done with
> > a different patch, and discussed in a different thread, so let's
> > drop this topic FTM (in the meantime, I removed the addition of
> > my name to credits in `instspc.sh').
> 
> I am *very* reluctant to change *any* author tags that are not
> mine. IMVHO, one would at the very least have to ask the
> corresponding author for permission.  And honestly, I'm not going
> to ask anybody.  I'm just glad there are almost no tags in the
> tests/ code today, and hoping that we can agree to not add more. 
Sure.  For what concerns credit for authors, contributors and bug
reporters, ChangeLog entries and git commit messages  are the way
to go.
> Again, if you would like to add your name to this file in that
> case, I won't prevent you from doing so.
No, the "everyone or no one" was just a preference, not a strong
desire or a rule.
 
> I consider git log information the most valuable meta data carrying
> authorship anyway.
Definitely.

> > > > +# This file will be sourced by many tests, so avoid
> > > > cluttering up +# the verbose logs too much.
> > > > +set +x
> > > 
> > > I understand, and agree, with the sentiment.  Problem however
> > > is that this will make it harder to debug your shell script,
> > > because
> > > 
> > >   $ sh -x instspc-$foo.test
> > > 
> > > will be useless.  Not sure how to avoid that.
> > 
> > Probably by analizying `$-' in tests/defs, and define proper
> > variables $disable_xtraces and $enable_xtraces.  But that's for
> > a follow-up patch IMHO (good material for the branch on
> > `tests/defs.in' refactoring, BTW).
> 
> Until there's a bug in $enable_xtraces code ...
A bug in somehing like this?
  case $- in *x*) disable_xtraces=':'; *) disable_xtraces='set +x';; esac
> (please try not to over-engineer things; there are lots of
> non-testsuite bugs also still waiting to be fixed)
That was for a follow-up patch anyway -- and now it's a moot point,
given the nature of the new code.
 
> > > > +echo "INFO: $me: shell traces disabled"
> > > 
> > > This weird "INFO: " prefix doesn't look like normal GCS-like
> > > messages.
> > 
> > This prefix is there so that the info messages stick out more
> > clearly among the tons of verbose logs. IMO we might think about
> > a refactoring of this later (maybe a new function in
> > `tests/defs.in'?).
> 
> I really dislike arms races.  Messages sticking out more clearly
> than others are an arms race, especially since the above message
> often will have very little value.  Please just use the normal GCS
> approach of prefixing the program name; that way, you can grep for
> what you are interested in *at the time* you are looking at a
> trace, not at what you thought users would be interested in when
> writing the code.
You have a point.  Anyway, this too is moot now.
 
> Which is a good sign that maybe the approach should be
> reconsidered. The code should be getting simpler and more
> maintainable, not the other way round.
> 
> > Here is the new much simplified code:
> Looks better, thanks, but I still wonder why you don't just do
> something like the following, and get rid of all eval and globbing
> and other complications:
> 
> ht=...
> first=:
> # loop over pairs: verbose name of characters, then characters.
> for chars in \
>     space ' ' htab "$ht" ...
> do
>   if $first; then
>     first=false
>     name=$chars
>     continue
>   else
>     first=:
>   fi
>   create test instspc-$name.test using $chars ...
> done
I SAW THE LIGHT! :-)

> That way, the code could become a lot simpler, the tests shorter,
> you wouldn't need to use 'set +x' any more, plus a lot of other
> nits above would just go away by itself, and a lot of the info
> messages could be omitted.  :-)
You are so right!

Thanks,
  Stefano

From f40cf3adb179ef443587173fa227008de3692509 Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Sun, 6 Jun 2010 18:38:27 +0200
Subject: [PATCH] Overhauled and modularized tests in `instspc.test'.

The test `instspc.test' was way too big and fragile.  Its running
time was very long.  It also produced a log that was nearly
unreadable due to its length, making it very difficult to find
out the reason for failures.
Also, it was too much monolithic, with a single (maybe spurious)
failure in a corner case causing the whole test to fail (even if
everything worked as expected in the other 99% of cases).
The present change should solve these problems, by separating
`instspc.test' into many smaller, self-contained, auto-generated
tests.

* tests/instspc.test: Removed.
* tests/instspc-tests.sh: New script, fullfilling a double role:
1. it generates a Makefile.am snippet `tests/instspc-tests.am',
containing the definition of a list of new tests which will take
over the older `instspc.test', and
2. it is sourced by said generated tests with proper parameters
pre-set, to run the "meat" of the checks.
This apparent abuse is indeed required because the test generation
code and test execution code are inevitably interwined.
* tests/Makefile.am ($(srcdir)/instspc-tests.am): Include this
snippet, which (among the other things) defines ...
(instspc_tests): ... this new macro, containing the list of the
newly generated `instspc*.test' tests, and ...
(instspc_xfail_tests): ... this new macro, containing the list
of the `instspc*.test' tests expected to fail.
($(instspc_tests)): New rule, generates the `instspc*.test' tests.
($(instspc_tests:.test=.log)): New rule, registers the dependency
of all `instspc*.test' tests on the `instspc-tests.sh' script.
(TESTS): Add `$(instspc_tests)', remove `instspc.test'.
(XFAIL_TESTS): Add `$(xfail_instspc_tests)'.
(EXTRA_DIST): Distribute instspc-tests.sh.
(MAINTAINERCLEANFILES): Added $(instspc_tests).
Other minor cosmetic changes.
* bootstrap: Generate instspc-tests.am.
* tests/.gitignore: Updated.
---
 ChangeLog              |   39 ++++++-
 bootstrap              |    3 +-
 tests/.gitignore       |    5 +-
 tests/Makefile.am      |   48 +++++++-
 tests/Makefile.in      |  110 ++++++++++++++++--
 tests/instspc-tests.sh |  299 ++++++++++++++++++++++++++++++++++++++++++++++++
 tests/instspc.test     |  187 ------------------------------
 7 files changed, 486 insertions(+), 205 deletions(-)
 create mode 100755 tests/instspc-tests.sh
 delete mode 100755 tests/instspc.test

diff --git a/ChangeLog b/ChangeLog
index 5ef4d4e..544456a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,4 +1,41 @@
-2010-09-14  Stefano Lattarini  <address@hidden>
+2010-09-16  Stefano Lattarini  <address@hidden>
+
+       Overhauled and modularized tests in `instspc.test'.
+       The test `instspc.test' was way too big and fragile.  Its running
+       time was very long.  It also produced a log that was nearly
+       unreadable due to its length, making it very difficult to find
+       out the reason for failures.
+       Also, it was too much monolithic, with a single (maybe spurious)
+       failure in a corner case causing the whole test to fail (even if
+       everything worked as expected in the other 99% of cases).
+       The present change should solve these problems, by separating
+       `instspc.test' into many smaller, self-contained, auto-generated
+       tests.
+       * tests/instspc.test: Removed.
+       * tests/instspc-tests.sh: New script, fullfilling a double role:
+       1. it generates a Makefile.am snippet `tests/instspc-tests.am',
+       containing the definition of a list of new tests which will take
+       over the older `instspc.test', and
+       2. it is sourced by said generated tests with proper parameters
+       pre-set, to run the "meat" of the checks.
+       This apparent abuse is indeed required because the test generation
+       code and test execution code are inevitably interwined.
+       * tests/Makefile.am ($(srcdir)/instspc-tests.am): Include this
+       snippet, which (among the other things) defines ...
+       (instspc_tests): ... this new macro, containing the list of the
+       newly generated `instspc*.test' tests, and ...
+       (instspc_xfail_tests): ... this new macro, containing the list
+       of the `instspc*.test' tests expected to fail.
+       ($(instspc_tests)): New rule, generates the `instspc*.test' tests.
+       ($(instspc_tests:.test=.log)): New rule, registers the dependency
+       of all `instspc*.test' tests on the `instspc-tests.sh' script.
+       (TESTS): Add `$(instspc_tests)', remove `instspc.test'.
+       (XFAIL_TESTS): Add `$(xfail_instspc_tests)'.
+       (EXTRA_DIST): Distribute instspc-tests.sh.
+       (MAINTAINERCLEANFILES): Added $(instspc_tests).
+       Other minor cosmetic changes.
+       * bootstrap: Generate instspc-tests.am.
+       * tests/.gitignore: Updated.
 
        Prefer `$(am__cd)' to plain `cd' in our Makefiles.
        * Makefile.am (recheck, dist-hook, git-dist, path-check, fetch)
diff --git a/bootstrap b/bootstrap
index 9f1a8d8..41f1f4e 100755
--- a/bootstrap
+++ b/bootstrap
@@ -103,9 +103,10 @@ dosubst m4/amversion.in m4/amversion.m4
 # Create temporary replacement for automake.
 dosubst automake.in automake.tmp
 
-# Create tests/parallel-tests.am.
+# Create required makefile snippets.
 cd tests
 $BOOTSTRAP_SHELL ./gen-parallel-tests > parallel-tests.am
+$BOOTSTRAP_SHELL ./instspc-tests.sh --generate-makefile > instspc-tests.am
 cd ..
 
 # Run the autotools.
diff --git a/tests/.gitignore b/tests/.gitignore
index 61b0783..ff89b40 100644
--- a/tests/.gitignore
+++ b/tests/.gitignore
@@ -1,9 +1,12 @@
 aclocal-*
 automake-*
 defs
+instspc-tests.am
 parallel-tests.am
 *.dir
 *.log
 *.log-t
 *-p.test
-*-p.test-t
+instspc-*-build.test
+instspc-*-install.test
+*.test-t
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 610b18c..ae9f3ba 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -16,6 +16,9 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+MAINTAINERCLEANFILES =
+EXTRA_DIST = ChangeLog-old
+
 XFAIL_TESTS = \
 all.test \
 auxdir2.test \
@@ -23,6 +26,7 @@ cond17.test \
 gcj6.test \
 txinfo5.test
 
+
 include $(srcdir)/parallel-tests.am
 
 $(srcdir)/parallel-tests.am: gen-parallel-tests Makefile.am
@@ -51,7 +55,44 @@ $(parallel_tests): Makefile.am
        } > address@hidden
        $(AM_V_at)chmod a+rx address@hidden && mv -f address@hidden $@
 
-MAINTAINERCLEANFILES = $(parallel_tests)
+MAINTAINERCLEANFILES += $(parallel_tests)
+EXTRA_DIST += gen-parallel-tests
+
+
+include $(srcdir)/instspc-tests.am
+
+$(srcdir)/instspc-tests.am: instspc-tests.sh Makefile.am
+       $(AM_V_GEN)($(am__cd) $(srcdir) \
+         && $(SHELL) ./instspc-tests.sh --generate-makefile) >$@
+
+$(instspc_tests): Makefile.am
+       $(AM_V_at)rm -f $@ address@hidden
+       $(AM_V_GEN) :; \
+         base=`expr x'$@' : x'instspc-\(.*\)\.test$$'`; \
+         name=`expr x"$$base" : x'\(.*\)-'`; \
+         action=`expr x"$$base" : x'.*-\(.*\)'`; \
+         { \
+           echo '#!/bin/sh'; \
+           echo '# DO NOT EDIT!  GENERATED AUTOMATICALLY!'; \
+           echo; \
+           echo '# Ensure proper definition of $$srcdir.'; \
+           echo 'am_skip_defs=yes'; \
+           echo '. ./defs || exit 99'; \
+           echo 'test -n "$$srcdir" || exit 99 # sanity check'; \
+           echo; \
+           echo "instspc_test_name='$$name'"; \
+           echo "instspc_action='test-$$action'"; \
+           echo ". \$$srcdir/instspc-tests.sh"; \
+         } > address@hidden
+       $(AM_V_at)chmod a+rx address@hidden && mv -f address@hidden $@
+
+# All instspc*.test tests work by sourcing the instspc-tests.sh script.
+$(instspc_tests:.test=.log): instspc-tests.sh
+
+MAINTAINERCLEANFILES += $(instspc_tests)
+EXTRA_DIST += instspc-tests.sh
+XFAIL_TESTS += $(instspc_xfail_tests)
+
 
 TESTS = \
 aclibobj.test \
@@ -412,7 +453,7 @@ instman2.test \
 instmany.test \
 instmany-mans.test \
 instmany-python.test \
-instspc.test \
+$(instspc_tests) \
 interp.test \
 interp2.test \
 java.test \
@@ -816,7 +857,8 @@ yflags.test \
 yflags2.test \
 $(parallel_tests)
 
-EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS)
+EXTRA_DIST += $(TESTS)
+
 
 # Each test case depends on defs, aclocal, and automake.
 $(TEST_LOGS): defs aclocal-$(APIVERSION) automake-$(APIVERSION)
diff --git a/tests/Makefile.in b/tests/Makefile.in
index c3c2863..9de466e 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -50,7 +50,7 @@ POST_UNINSTALL = :
 build_triplet = @build@
 DIST_COMMON = README $(srcdir)/Makefile.am $(srcdir)/Makefile.in \
        $(srcdir)/aclocal.in $(srcdir)/automake.in $(srcdir)/defs.in \
-       $(srcdir)/parallel-tests.am
+       $(srcdir)/instspc-tests.am $(srcdir)/parallel-tests.am
 subdir = tests
 ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
 am__aclocal_m4_deps = $(top_srcdir)/m4/amversion.m4 \
@@ -250,20 +250,79 @@ target_alias = @target_alias@
 top_build_prefix = @top_build_prefix@
 top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
-XFAIL_TESTS = \
-all.test \
-auxdir2.test \
-cond17.test \
-gcj6.test \
-txinfo5.test
-
+MAINTAINERCLEANFILES = $(parallel_tests) $(instspc_tests)
+EXTRA_DIST = ChangeLog-old gen-parallel-tests instspc-tests.sh \
+       $(TESTS)
+XFAIL_TESTS = all.test auxdir2.test cond17.test gcj6.test txinfo5.test \
+       $(instspc_xfail_tests)
 parallel_tests = check-p.test check10-p.test check11-p.test \
        check2-p.test check3-p.test check4-p.test check5-p.test \
        check6-p.test check7-p.test check8-p.test check9-p.test \
        color-p.test color2-p.test comment9-p.test dejagnu-p.test \
        exeext4-p.test maken3-p.test maken4-p.test pr401-p.test \
        pr401b-p.test pr401c-p.test
-MAINTAINERCLEANFILES = $(parallel_tests)
+instspc_tests = instspc-squote-build.test instspc-squote-install.test \
+       instspc-dquote-build.test instspc-dquote-install.test \
+       instspc-bquote-build.test instspc-bquote-install.test \
+       instspc-sharp-build.test instspc-sharp-install.test \
+       instspc-dollar-build.test instspc-dollar-install.test \
+       instspc-bang-build.test instspc-bang-install.test \
+       instspc-bslash-build.test instspc-bslash-install.test \
+       instspc-ampersand-build.test instspc-ampersand-install.test \
+       instspc-percent-build.test instspc-percent-install.test \
+       instspc-leftpar-build.test instspc-leftpar-install.test \
+       instspc-rightpar-build.test instspc-rightpar-install.test \
+       instspc-pipe-build.test instspc-pipe-install.test \
+       instspc-caret-build.test instspc-caret-install.test \
+       instspc-tilde-build.test instspc-tilde-install.test \
+       instspc-qmark-build.test instspc-qmark-install.test \
+       instspc-star-build.test instspc-star-install.test \
+       instspc-plus-build.test instspc-plus-install.test \
+       instspc-minus-build.test instspc-minus-install.test \
+       instspc-comma-build.test instspc-comma-install.test \
+       instspc-colon-build.test instspc-colon-install.test \
+       instspc-semicol-build.test instspc-semicol-install.test \
+       instspc-equal-build.test instspc-equal-install.test \
+       instspc-less-build.test instspc-less-install.test \
+       instspc-more-build.test instspc-more-install.test \
+       instspc-at-build.test instspc-at-install.test \
+       instspc-lqbrack-build.test instspc-lqbrack-install.test \
+       instspc-rqbrack-build.test instspc-rqbrack-install.test \
+       instspc-lcbrack-build.test instspc-lcbrack-install.test \
+       instspc-rcbrack-build.test instspc-rcbrack-install.test \
+       instspc-space-build.test instspc-space-install.test \
+       instspc-tab-build.test instspc-tab-install.test \
+       instspc-linefeed-build.test instspc-linefeed-install.test \
+       instspc-backspace-build.test instspc-backspace-install.test \
+       instspc-formfeed-build.test instspc-formfeed-install.test \
+       instspc-carriageret-build.test \
+       instspc-carriageret-install.test \
+       instspc-quadrigraph0-build.test \
+       instspc-quadrigraph0-install.test \
+       instspc-quadrigraph1-build.test \
+       instspc-quadrigraph1-install.test \
+       instspc-quadrigraph2-build.test \
+       instspc-quadrigraph2-install.test \
+       instspc-quadrigraph3-build.test \
+       instspc-quadrigraph3-install.test \
+       instspc-quadrigraph4-build.test \
+       instspc-quadrigraph4-install.test instspc-a_b-build.test \
+       instspc-a_b-install.test instspc-a__b-build.test \
+       instspc-a__b-install.test instspc-a_lf_b-build.test \
+       instspc-a_lf_b-install.test instspc-dotdotdot-build.test \
+       instspc-dotdotdot-install.test instspc-dosdrive-build.test \
+       instspc-dosdrive-install.test instspc-miscglob1-build.test \
+       instspc-miscglob1-install.test instspc-miscglob2-build.test \
+       instspc-miscglob2-install.test
+instspc_xfail_tests = instspc-squote-build.test \
+       instspc-dquote-build.test instspc-bquote-build.test \
+       instspc-sharp-build.test instspc-dollar-build.test \
+       instspc-bslash-build.test instspc-ampersand-build.test \
+       instspc-linefeed-build.test instspc-quadrigraph0-build.test \
+       instspc-a_lf_b-build.test instspc-squote-install.test \
+       instspc-dquote-install.test instspc-bquote-install.test \
+       instspc-sharp-install.test instspc-dollar-install.test \
+       instspc-linefeed-install.test instspc-a_lf_b-install.test
 TESTS = \
 aclibobj.test \
 aclocal.test \
@@ -623,7 +682,7 @@ instman2.test \
 instmany.test \
 instmany-mans.test \
 instmany-python.test \
-instspc.test \
+$(instspc_tests) \
 interp.test \
 interp2.test \
 java.test \
@@ -1027,12 +1086,11 @@ yflags.test \
 yflags2.test \
 $(parallel_tests)
 
-EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS)
 all: all-am
 
 .SUFFIXES:
 .SUFFIXES: .html .log .test
-$(srcdir)/Makefile.in:  $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am 
$(am__configure_deps)
+$(srcdir)/Makefile.in:  $(srcdir)/Makefile.am $(srcdir)/parallel-tests.am 
$(srcdir)/instspc-tests.am $(am__configure_deps)
        @for dep in $?; do \
          case '$(am__configure_deps)' in \
            *$$dep*) \
@@ -1426,6 +1484,34 @@ $(parallel_tests): Makefile.am
        } > address@hidden
        $(AM_V_at)chmod a+rx address@hidden && mv -f address@hidden $@
 
+$(srcdir)/instspc-tests.am: instspc-tests.sh Makefile.am
+       $(AM_V_GEN)($(am__cd) $(srcdir) \
+         && $(SHELL) ./instspc-tests.sh --generate-makefile) >$@
+
+$(instspc_tests): Makefile.am
+       $(AM_V_at)rm -f $@ address@hidden
+       $(AM_V_GEN) :; \
+         base=`expr x'$@' : x'instspc-\(.*\)\.test$$'`; \
+         name=`expr x"$$base" : x'\(.*\)-'`; \
+         action=`expr x"$$base" : x'.*-\(.*\)'`; \
+         { \
+           echo '#!/bin/sh'; \
+           echo '# DO NOT EDIT!  GENERATED AUTOMATICALLY!'; \
+           echo; \
+           echo '# Ensure proper definition of $$srcdir.'; \
+           echo 'am_skip_defs=yes'; \
+           echo '. ./defs || exit 99'; \
+           echo 'test -n "$$srcdir" || exit 99 # sanity check'; \
+           echo; \
+           echo "instspc_test_name='$$name'"; \
+           echo "instspc_action='test-$$action'"; \
+           echo ". \$$srcdir/instspc-tests.sh"; \
+         } > address@hidden
+       $(AM_V_at)chmod a+rx address@hidden && mv -f address@hidden $@
+
+# All instspc*.test tests work by sourcing the instspc-tests.sh script.
+$(instspc_tests:.test=.log): instspc-tests.sh
+
 # Each test case depends on defs, aclocal, and automake.
 $(TEST_LOGS): defs aclocal-$(APIVERSION) automake-$(APIVERSION)
 
diff --git a/tests/instspc-tests.sh b/tests/instspc-tests.sh
new file mode 100755
index 0000000..4004575
--- /dev/null
+++ b/tests/instspc-tests.sh
@@ -0,0 +1,299 @@
+#! /bin/sh
+# Copyright (C) 2010 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+#
+# Driver script to generate and run tests checking that building from,
+# or installing to, directories with shell metacharacters succeed.
+#
+# Original report from James Amundson about file names with spaces.
+# Other characters added by Paul Eggert.
+#
+# This script fullfills a double role:
+#   1. It generates a Makefile.am snippet, containing the definintion
+#      of proper lists of tests.
+#   2. It is sourced by said generated tests with proper parameters
+#      pre-set, to run the "meat" of the checks.
+# This setup might seem tricky and over-engineered abuse, but past
+# (painful) experiences showed that it is indeed required, because
+# the test generation code and test execution code tend to be
+# inextricably coupled and interwined.
+#
+
+# Be more Bourne compatible (snippet copied from `tests/defs.in').
+DUALCASE=1; export DUALCASE # for MKS sh
+if test -n "${ZSH_VERSION+set}" && (emulate sh) >/dev/null 2>&1; then
+  emulate sh
+  NULLCMD=:
+  # Pre-4.2 versions of Zsh do word splitting on ${1+"$@"}, which
+  # is contrary to our usage.  Disable this feature.
+  alias -g '${1+"$@"}'='"$@"'
+  setopt NO_GLOB_SUBST
+else
+  case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac
+fi
+
+set -e
+
+# Sanity and usage checks.
+if test x"$instspc_action" = x; then
+  if test "$#,$1" = "1,--generate-makefile"; then
+    instspc_action=generate-makefile
+  else
+    echo "$0: empty action and no proper command line " >&2
+    exit 99
+  fi
+elif test $# -gt 0; then
+  echo "$0: action specified and command line arguments used" >&2
+  exit 99
+elif test x"$instspc_action" = x"generate-makefile"; then
+    :
+else
+  case $instspc_action in
+    test-build|test-install)
+      if test x"$instspc_test_name" = x; then
+        echo "$0: test name undefined for action '$instspc_action'" >&2
+        exit 99
+      fi;;
+    *)
+      echo "$0: invalid action: '$instspc_action'"
+      exit 99;;
+  esac
+fi
+
+# Helper subroutine for test data definition.
+# Usage: define_problematic_string NAME STRING
+define_problematic_string ()
+{
+  tst=$1
+  shift
+  eval "instspc__$tst=\$1" || exit 99
+  shift
+  instspc_names_list="$instspc_names_list $tst"
+  # Some of the "problematic" characters cannot be used in the name of
+  # a build or install directory on a POSIX host.  These lists should
+  # be empty, but are not due to limitations in Autoconf, Automake, Make,
+  # M4, or the shell.
+  case " $* " in *' fail-build '*|*' build-fail '*)
+    instspc_xfail_builds_list="$instspc_xfail_builds_list $tst";;
+  esac
+  case " $* " in *' fail-install '*|*' install-fail '*)
+    instspc_xfail_installs_list="$instspc_xfail_installs_list $tst";;
+  esac
+}
+
+# Be sure to avoid interferences from the environment.
+instspc_names_list=''
+instspc_xfail_builds_list=''
+instspc_xfail_installs_list=''
+
+
+# ================= #
+#  Test data begin  #
+# ----------------- #
+
+# Some control characters that are white space.
+bs=''   # back space
+cr='
'   # carriage return
+ff=''   # form feed
+ht='   ' # horizontal tab
+lf='
+'         # line feed (aka newline)
+
+# Hack to save typing and make code visually clearer.
+def=define_problematic_string
+
+$def    squote          \'          fail-build  fail-install
+$def    dquote          '"'         fail-build  fail-install
+$def    bquote          '`'         fail-build  fail-install
+$def    sharp           '#'         fail-build  fail-install
+$def    dollar          '$'         fail-build  fail-install
+$def    bang            '!'
+$def    bslash          '\'         fail-build
+$def    ampersand       '&'         fail-build
+$def    percent         '%'
+$def    leftpar         '('
+$def    rightpar        ')'
+$def    pipe            '|'
+$def    caret           '^'
+$def    tilde           '~'
+$def    qmark           '?'
+$def    star            '*'
+$def    plus            '+'
+$def    minus           '-'
+$def    comma           ','
+$def    colon           ':'
+$def    semicol         ';'
+$def    equal           '='
+$def    less            '<'
+$def    more            '>'
+$def    at              '@'
+$def    lqbrack         '['
+$def    rqbrack         ']'
+$def    lcbrack         '{'
+$def    rcbrack         '}'
+$def    space           ' '
+$def    tab             "$ht"
+$def    linefeed        "$lf"       fail-build  fail-install
+$def    backspace       "$bs"
+$def    formfeed        "$ff"
+$def    carriageret     "$cr"
+$def    quadrigraph0    '@&t@'      fail-build
+$def    quadrigraph1    '@<:@'
+$def    quadrigraph2    '@:>@'
+$def    quadrigraph3    '@S|@'
+$def    quadrigraph4    '@%:@'
+$def    a_b             'a b'
+$def    a__b            'a  b'
+$def    a_lf_b          "a${lf}b"   fail-build  fail-install
+$def    dotdotdot       '...'
+$def    dosdrive        'a:'
+$def    miscglob1       '?[a-z]*'
+$def    miscglob2       '.*?[0-9]'
+
+unset def
+
+# --------------- #
+#  Test data end  #
+# =============== #
+
+
+if test x"$instspc_action" = x"generate-makefile"; then
+  # We must generate a makefile fragment on stdout.  It must refer
+  # to all tests at once, hence the loop below.
+  echo '## Generated by instspc-tests.sh.  DO NOT EDIT!'
+  echo 'instspc_tests ='
+  echo 'instspc_xfail_tests ='
+  for test_name in $instspc_names_list; do
+    echo "instspc_tests += instspc-$test_name-build.test"
+    echo "instspc_tests += instspc-$test_name-install.test"
+  done
+  for test_name in $instspc_xfail_builds_list; do
+    echo "instspc_xfail_tests += instspc-$test_name-build.test"
+  done
+  for test_name in $instspc_xfail_installs_list; do
+    echo "instspc_xfail_tests += instspc-$test_name-install.test"
+  done
+  exit 0
+fi
+
+###  If we are still here, we have to run a test ...
+
+# We'll need the full setup provided by `tests/defs'.
+. ./defs || Exit 99
+
+eval "instspc_test_string=\${instspc__$instspc_test_name}" || Exit 99
+if test x"$instspc_test_string" = x; then
+  echo "$me: invalid test name: '$instspc_test_name'" >&2
+  Exit 99
+fi
+
+# Skip if this system doesn't support these characters in file names.
+mkdir "./$instspc_test_string" || Exit 77
+
+mkdir sub1
+
+cat >> configure.in << 'EOF'
+AC_PROG_CC
+AC_PROG_RANLIB
+AC_OUTPUT
+EOF
+
+mkdir sub
+
+: > sub/base.h
+: > sub/nobase.h
+: > sub/base.dat
+: > sub/nobase.dat
+: > sub/base.sh
+: > sub/nobase.sh
+
+cat > source.c << 'EOF'
+int
+main (int argc, char **argv)
+{
+  return 0;
+}
+EOF
+cp source.c source2.c
+
+cat > Makefile.am << 'EOF'
+foodir = $(prefix)/foo
+fooexecdir = $(prefix)/foo
+
+foo_HEADERS = sub/base.h
+nobase_foo_HEADERS = sub/nobase.h
+
+dist_foo_DATA = sub/base.dat
+nobase_dist_foo_DATA = sub/nobase.dat
+
+dist_fooexec_SCRIPTS = sub/base.sh
+nobase_dist_fooexec_SCRIPTS = sub/nobase.sh
+
+fooexec_PROGRAMS = sub/base
+nobase_fooexec_PROGRAMS = sub/nobase
+sub_base_SOURCES = source.c
+sub_nobase_SOURCES = source.c
+
+fooexec_LIBRARIES = sub/libbase.a
+nobase_fooexec_LIBRARIES = sub/libnobase.a
+sub_libbase_a_SOURCES = source.c
+sub_libnobase_a_SOURCES = source.c
+
+.PHONY: test-install-sep
+test-install-sep: install
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.h'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.h'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.h'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.dat'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.dat'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.dat'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.sh'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.sh'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.sh'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase$(EXEEXT)'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase$(EXEEXT)'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/base$(EXEEXT)'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/libnobase.a'
+       test ! -f '$(DESTDIR)/$(file)-prefix/foo/libnobase.a'
+       test   -f '$(DESTDIR)/$(file)-prefix/foo/libbase.a'
+EOF
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+case $instspc_action in
+  test-build)
+    build=$instspc_test_string
+    dest=`pwd`/sub1
+    ;;
+  test-install)
+    build=sub1
+    dest=`pwd`/$instspc_test_string
+    ;;
+  *)
+    echo "$me: internal error: invalid action '$instspc_action'"
+    Exit 99
+    ;;
+esac
+
+cd "./$build"
+
+../configure --prefix "/$instspc_test_string-prefix"
+$MAKE
+DESTDIR="$dest" file="$instspc_test_string" $MAKE -e test-install-sep
+
+:
diff --git a/tests/instspc.test b/tests/instspc.test
deleted file mode 100755
index 414b3e5..0000000
--- a/tests/instspc.test
+++ /dev/null
@@ -1,187 +0,0 @@
-#! /bin/sh
-# Copyright (C) 2004, 2005  Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-# Check that installation to directory with shell metacharacters succeed.
-# Original report from James Amundson about file names with spaces.
-# Other characters added by Paul Eggert.
-
-# This is mostly the same input as nobase.test, but we do not use
-# libtool libraries, because Libtool does not preserve space in
-# file names (Issue observed with ltmain.sh (GNU libtool) 1.5a (1.1323
-# 2003/11/10 21:06:47))
-
-
-required='gcc'
-. ./defs || Exit 1
-
-set -e
-
-# Set up files that won't change each time through the loop.
-
-cat >> configure.in <<'EOF'
-AC_PROG_CC
-AC_PROG_RANLIB
-AC_OUTPUT
-EOF
-
-mkdir sub
-
-: > sub/base.h
-: > sub/nobase.h
-: > sub/base.dat
-: > sub/nobase.dat
-: > sub/base.sh
-: > sub/nobase.sh
-
-cat >source.c <<'EOF'
-int
-main (int argc, char **argv)
-{
-  return 0;
-}
-EOF
-cp source.c source2.c
-
-cat > Makefile.am << 'EOF'
-foodir = $(prefix)/foo
-fooexecdir = $(prefix)/foo
-
-foo_HEADERS = sub/base.h
-nobase_foo_HEADERS = sub/nobase.h
-
-dist_foo_DATA = sub/base.dat
-nobase_dist_foo_DATA = sub/nobase.dat
-
-dist_fooexec_SCRIPTS = sub/base.sh
-nobase_dist_fooexec_SCRIPTS = sub/nobase.sh
-
-fooexec_PROGRAMS = sub/base
-nobase_fooexec_PROGRAMS = sub/nobase
-sub_base_SOURCES = source.c
-sub_nobase_SOURCES = source.c
-
-fooexec_LIBRARIES = sub/libbase.a
-nobase_fooexec_LIBRARIES = sub/libnobase.a
-sub_libbase_a_SOURCES = source.c
-sub_libnobase_a_SOURCES = source.c
-
-test-install-sep: install
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.h'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.h'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.h'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.dat'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.dat'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.dat'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase.sh'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase.sh'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base.sh'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/nobase$(EXEEXT)'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/nobase$(EXEEXT)'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/base$(EXEEXT)'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/sub/libnobase.a'
-       test ! -f '$(DESTDIR)/$(file)-prefix/foo/libnobase.a'
-       test   -f '$(DESTDIR)/$(file)-prefix/foo/libbase.a'
-EOF
-
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE -a
-
-# Some control characters that are white space:
-# back space, carriage return, form feed, horizontal tab, line feed, space
-bs=''
-cr='
'
-ff=''
-ht='   '
-lf='
-'
-sp=' '
-
-build_failures=
-install_failures=
-
-for file in \
-  '!' '"' '#' '$' '%' '&' \' '(' ')' '*' '+' ',' '-' ':' ';' \
-  '<' '=' '>' '?' '@' '[' '\' ']' '^' '`' '{' '|' '}' '~' \
-  "$bs" "$cr" "$ff" "$ht" "$lf" "$sp" \
-  '@<:@' '@:>@' '@S|@' '@%:@' '@&t@' \
-  "a${sp}b" "a${sp}${sp}b" "a${lf}b" ... a:
-do
-  for test in build install; do
-    case $test in
-    build)
-      build=$file
-      dest=`pwd`/sub1;;
-    install)
-      build=sub1
-      dest=`pwd`/$file;;
-    esac
-
-    # Make sure this system supports this character in file names.
-    mkdir sub1 "./$file" || Exit 77
-
-    cd "$build"
-
-    ../configure --prefix "/$file-prefix" &&
-    $MAKE &&
-    DESTDIR=$dest file=$file $MAKE -e test-install-sep ||
-      eval "${test}_failures=\"\$${test}_failures$lf\$file\""
-
-    cd ..
-
-    rm -fr sub1 "./$file"
-  done
-done
-
-# The list of the above file names that cannot be used as a build directory
-# on a POSIX host.  This list should be empty, but is not due to limitations
-# in Autoconf, Automake, Make, M4, or the shell.
-expected_build_failures='
-"
-#
-$
-&
-'\''
-\
-`
-'"$lf"'
-@&t@
-a'"${lf}"'b'
-
-# Similarly, the list of file names that cannot be used as an install directory
-# on a POSIX host.  This list should also be empty.
-expected_install_failures='
-"
-#
-$
-'\''
-`
-'"$lf"'
-a'"${lf}"'b'
-
-fail=0
-for test in build install; do
-  eval failures=\$${test}_failures
-  case $failures in
-  ?*)
-    cat >&2 <<EOF
-$0: $test test failed for the following file names:$failures
-EOF
-    eval test \"\$failures\" = \"\$expected_${test}_failures\" || fail=1
-  esac
-done
-
-Exit $fail
-- 
1.7.1