[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Excluding FAILs from UNSUPPORTED test cases
From: |
Arsen Arsenović |
Subject: |
Re: Excluding FAILs from UNSUPPORTED test cases |
Date: |
Fri, 23 Sep 2022 15:02:13 +0200 |
Hi,
Proceeding with the ${tool}-dg-prune based solution, I came across a
test that does not emit PASS+UNSUPPORTED, and FAIL+UNSUPPORTED instead,
so I went digging. It would appear that, when ${tool}-dg-prune is used
to implement this unsupported test, the reason we got PASSes as well as
UNSUPPORTED is error recovery: the special #error we were using to mark
unsupported tests would get ignored, and the compiler would resume;
however, this is a problem in some tests, because it's not always
possible to get good results when that #error is emitted; and indeed,
running with -Wfatal-errors produces a plethora of FAILs.
On Friday, 23 September 2022 12:02:56 CEST Jonathan Wakely wrote:
> But it seems to
> me that ideally the individual checks would get "retroactively
> skipped" if tool-dg-prune returns one of ::untested::, ::unresolved::,
> or ::unsupported::. Maybe that's not easy to do though.
The easiest way to achieve this is AFAICT to allow ${tool}-dg-test to
return a sentinel value that would prevent processing of any further
FAILs/PASSes by returning from the testcase immediately after the
${tool}-dg-test call (maybe a llength == 1 list whose first element is
one of ::{unsupported,untested,unresolved}::$msg).
Thanks,
--
Arsen Arsenović
signature.asc
Description: This is a digitally signed message part.
- Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/22
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/22
- Re: Excluding FAILs from UNSUPPORTED test cases, Jonathan Wakely, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases, Jonathan Wakely, 2022/09/23
- Re: Excluding FAILs from UNSUPPORTED test cases,
Arsen Arsenović <=
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/24
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/24
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/25
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/26
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/29
- Re: Excluding FAILs from UNSUPPORTED test cases, Arsen Arsenović, 2022/09/30
- Re: Excluding FAILs from UNSUPPORTED test cases, Jacob Bachmeyer, 2022/09/30