Re: [PATCH] {master} Extend, fix and improve tests on Lex and Yacc support.

[Ralf Wildenhues]

* Stefano Lattarini wrote on Fri, Dec 17, 2010 at 03:33:32PM CET:
> On Friday 17 December 2010, Ralf Wildenhues wrote:
> > 
> > The *current* tests are good enough for the current code.
> >
> Yes, but not enough if we plan to modify that code

You can add tests as you go modifying the code.  Doing that is very
helpful and instructive actually, because while you focus on the code,
it is easier to have a very clear picture of what invariants need to be
tested.

> (well, it depends on
> which part of the code we are referring, obviously; some parts are
> excellently tested, some decently, some scarcely or nigh to nothing).
> This is especially true in case heavy refactorings are to be done (not
> that I plan/want to do them ATM, but you never know).

Oh, heavy refactoring is a completely different beast.  For that, it
would be better to have a clear design goal first, then add specific
test coverage for affected bits and work on the implementation.  Here,
some more back and forth may be needed.  I don't see the past testsuite
work to be tailored towards that at all though.

I also don't see how "you never know" works.  I don't program by
accident ...

> > How many bugs have we found due to the testsuite frenzy in the last few
> > months with its 200+ patches?  I'd guess less than 5.
> >
> Maybe even less.  But to be honest, that was not the point of my
> testsuite patches; what I was really aiming for was to make it difficult
> to introduce in the future *new* bugs in the already-working paths of
> the code.

Honestly: how much do you think you have made it less difficult?
Honestly: which of the bugs that the new tests will catch do you
think you would have made inadvertently without having the tests?

> > That's an awful signal to noise ratio.  How much have you gotten to
> > know the Automake code base (outside of tests/) as a result of that?
> >
> Not much (apart from the parts I had to understand to provide my few
> non-tetsuite related patches).  But I've come to know quite well many
> automake APIs, interfaces, limitations & tricks, and many portability
> problems of various shells, utilities and make implementations.  This
> knowledge is IMHO a necessary (not sufficient!) prerequisite for any
> half-serious automake hacking.

Good!  That's great.

> > > > It would be really nice if somebody who knows their ways around
> > > > bison/yacc and flex/lex well
> > > >
> > > I'm not that somebody, unfortunately.
> > 
> > Don't give yourself up before starting.
> >
> Well, that wasn't a "giving up", just the stating of a fact: that
> my *current* Lex/Yacc fu is pratically non-existent.  Not that this
> fact can't be changed with some trying and sweating ...

Hehe.

> > > > >  # Don't redefine several times the same variable.
> > > > > -cp Makefile.am Makefile.src
> > > > > +cp -f Makefile.am Makefile.src
> > > > 
> > > > Why should you need this change?  Generally, I don't see why you ever
> > > > need 'cp -f'.
> > > >
> > > I dimly remember that I used to think there were cp implementations which
> > > might prompt if stdout/stderr is a tty and the dest file exists, unless
> > > the `-f' option is used..
> > 
> > That is true, but when you run a test, there is no tty involved.

That part (about the tty) is not true.  Sorry about that.

> Even when I run it from the shell prompt?

cp should only ever prompt when given -i.  That may happen through a
shell alias.  But aliases are not normally enabled in scripts.

If and when we find a cp that does prompt from a script, then we can
think of going back to this; but only after amending autoconf.texi
(which, by the way, also documents that some old cp versions do not
accept -f at all).

Cheers,
Ralf