Re: emacs-29 9c0d7bb73b 2/2: Add automated tests for Eglot

[Stefan Kangas]

Michael Albinus <michael.albinus@gmx.de> writes:

>> I'd basically consider any test over half a second or a second or so as
>> a candidate for being marked as :expensive.  But we should do it
>> judiciously, of course.  (And even better if we can find a way to make
>> slow tests faster.)
>
> That's a very restrictive rule. IIRC, we've discussed this a while ago,
> and a recommendation was to regard tests lasting more than 10 seconds
> being expensive. But I might remeber this wrong.

It's just a rule of thumb that I use to decide which tests to consider
marking.  We should of course not do that in a blanket fashion, because
we really do want some tests to always run, even if they take a long
time.

But the value of the test suite is decreased if it is slow by default,
as people will avoid running it.  We are already seeing a not
insignificant number of new test failures regularly introduced on master
and the release branch.  This suggests to me that this might already be
taking place.

(When dealing with automatic merging, as I do, a broken test suite is a
nuisance.  I don't have any hard data unfortunately, but over the last
3-6 months it feels to me like the test suite has been broken more often
than it's not.  Perhaps this observation could be checked against the
EMBA logs or similar.)

>>    passed  12/50  eglot--tramp-test (2.158451 sec)
>
> All eglot Tramp tests could be regarded as :expensive by default, I
> guess.

I don't know enough about Tramp to have an opinion, myself.  But please
do it if you think it makes sense, thanks.