Re: [PATCH 3/3] iotests: distinguish 'skipped' and 'not run' states

[Denis V. Lunev]

On 9/12/23 22:03, Vladimir Sementsov-Ogievskiy wrote:
> On 06.09.23 17:09, Denis V. Lunev wrote:
> > Each particular testcase could skipped intentionally and accidentally.
> > For example the test is not designed for a particular image format or
> > is not run due to the missed library.
> >
> > The latter case is unwanted in reality. Though the discussion has
> > revealed that failing the test in such a case would be bad. Thus the
> > patch tries to do different thing. It adds additional status for
> > the test case - 'skipped' and bound intentinal cases to that state.
>
> Hmm. Do I miss something, or in this patch you only split them, not 
> making "not run" produce an error? So ./check still reports success 
> when some tests are "not run"?
>
> The split itself looks correct to me.
The original talk was to avoid failing of such tests.
If we would let them fail - that could be done much
faster and without dances.

Thus tests are still counted as "skipped other way".
But I would definitely like to concentrate my attention
on something abnormal, i.e. things that should be
run but don't run.

These tests count be missed due to error (see patch 1)
or due to the missed package. And the reason of skip
is barely visible within the list of 100+ tests. While
1-2-3 "should not be skipped" tests are clearly in the
focus.

Den