Re: [PATCH 3/3] iotests: distinguish 'skipped' and 'not run' states

[Eric Blake]

On Wed, Sep 13, 2023 at 10:36:35AM -0500, Eric Blake wrote:
> On Wed, Sep 06, 2023 at 04:09:17PM +0200, Denis V. Lunev wrote:
> > Each particular testcase could skipped intentionally and accidentally.
> > For example the test is not designed for a particular image format or
> > is not run due to the missed library.
> > 
> > The latter case is unwanted in reality. Though the discussion has
> > revealed that failing the test in such a case would be bad. Thus the
> > patch tries to do different thing. It adds additional status for
> > the test case - 'skipped' and bound intentinal cases to that state.
> 
> intentional
> 
> > 
> > Signed-off-by: Denis V. Lunev <den@openvz.org>
> > CC: Kevin Wolf <kwolf@redhat.com>
> > CC: Hanna Reitz <hreitz@redhat.com>
> > CC: Eric Blake <eblake@redhat.com>
> > CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
> > ---
> >  tests/qemu-iotests/common.rc     | 23 ++++++++++++++++-------
> >  tests/qemu-iotests/iotests.py    | 19 +++++++++++++++----
> >  tests/qemu-iotests/testrunner.py | 17 +++++++++++++++--
> >  3 files changed, 46 insertions(+), 13 deletions(-)
> 
> I agree that this makes it easier to identify skipped tests that can
> never be run (not designed for that combination) vs tests that can
> plausibly be run by addressing missing dependencies (such as
> installing additional libraries, a different file system, ...).
> 
> Reviewed-by: Eric Blake <eblake@redhat.com>
> 
> Because patch 1 impacts NBD testing, I can queue the series through my
> NBD tree if no one else picks it up first.

Given further conversation, I've removed patch 3 from my queue while
conversation continues, but patch 1 and 2 are still queued (they are
independently useful, whether or not we add another result category)

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization:  qemu.org | libguestfs.org