bug-make
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Enhancements to the regression tests


From: Paul Smith
Subject: Enhancements to the regression tests
Date: Mon, 03 Apr 2023 08:26:21 -0400
User-agent: Evolution 3.48.0 (by Flathub.org)

Hi all; I reworked the behavior of the regression tests in the
following way:

Each test file has a separate subdirectory created for it in work/ and
before the test starts the driver will make that the current working
directory, then switch back again after the test completes.

The idea is to prevent one set of tests from interfering with another,
by accidentally leaving files lying around etc. and also to ensure that
if someone kills a regression test run in the middle, it doesn't leave
extra files around that might break future test runs.

I considered trying to create a separate directory per test but that
would have required rewriting a lot of tests.

There is still a bit of cleanup I need to do here in terms of removing
unused directories (if you skip a set of tests they're not removed). 
In the future when writing tests I recommend that we (a) try to use
different "extra" files for each test rather than re-using the same
ones, and (b) do NOT clean them up after the test completes.

The cleanup is no longer needed to prevent other test files from seeing
them, and leaving them behind allows us to more easily re-run that test
by hand (say, invoke make under a debugger etc.)

I did not try to change any existing tests to avoid removing files.

You can also now run the tests from an out-of-tree build without
needing weird symlink behavior.



reply via email to

[Prev in Thread] Current Thread [Next in Thread]