Re: mark expensive tests

[Michael Albinus]

John Wiegley <address@hidden> writes:

>> make check
>> make check-expensive
>> make check-very-expensive
>
> Prior art from an existing GNU project is even better. Now, who wants to make
> it so?

I could start with it, in the emacs-25 branch. Those targets do cover #2,
#3 and #4 tests. Which make target shall we use for #1? "make check-smoke"?

Also, we need to decide how to distinguish such test categories. I don't
believe we shall do it on test file basis, except maybe for the smoke
tests. For the other tests, we could request something like this in the
test functions:

#3 tests: (skip-unless (or (getenv "RUN_EXPENSIVE_TESTS")
                           (getenv "RUN_VERY_EXPENSIVE_TESTS")))

#4 tests: (skip-unless (getenv "RUN_VERY_EXPENSIVE_TESTS"))

Alternatively, we could extend ert.el by a mechanism, marking tests
optionally as "smoke", "normal", "expensive", "very-expensive". "normal"
would be the default category. This shall be in accordance to the GNU
Coding Standards, as Richard proposes.

Best regards, Michael.