Re: Policy about not-yet-regression tests?

[David Kastrup]

Graham Percival <address@hidden> writes:

> On Sat, Apr 09, 2011 at 07:28:00AM +0200, David Kastrup wrote:
>> How do I notice that somebody fixed the bug by accident if I am not
>> allowed to add a check for the bug to the regtests?
>> 
>> Am I supposed to keep a _private_ regtest for bugs I report, checking it
>> periodically to make sure that the bug is still there?
>> 
>> That makes me a single point of failure.  I am not confident that this
>> is a good idea...
>
> I'm not confident that it's a good idea either, but I'm not
> confident that I really understand the idea at all.

We have made improvements in typesetting.  Those improvements show up
when you do "make check" between versions.  The idea was that since
slight improvements show up anyway, why not have large improvements aka
as deficiency fixes show up as well?

> You are concerned that we accidentally fix bugs so often that you
> would like an automatic method of checking for those bug fixes?

I don't want deficiencies hidden away in a separate offline database
until they have been fixed.

Perhaps a different directory/label is a reasonable idea, and once a bug
has been fixed, one can elevate the respective test to the status of
"regtest".

We have a standard problem "bugfix should have regtest" which is not
working all too well.  If the responsibilities for bugfix and regtest
get decoupled, this might help.  After all, most bug reports already
require a demonstration to get accepted, and this demonstration will
often serve perfectly well as such a test.  Getting it into Lilypond
tells the bug reporter "something has been taken seriously".

Bouncing the status to "regtest" from "bug demonstration" can be done by
somebody actually verifying the fix rather than the fix committer.  Sort
of a frog task.

-- 
David Kastrup