|
From: | Jacob Bachmeyer |
Subject: | Re: Excluding FAILs from UNSUPPORTED test cases |
Date: | Thu, 22 Sep 2022 22:33:42 -0500 |
User-agent: | Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.22) Gecko/20090807 SeaMonkey/1.1.17 Mnenhy/0.7.6.0 |
Arsen Arsenović wrote:
Hi,While working on freestanding libstdc++, we came up with the idea to use a special error message to detect tests that use headers that don't work in freestanding, and mark them as unsupported by emitting ::unsupported:: from libstdc++-dg-test.
I have not yet had the occasion to dive into the dg.exp code, but a quick look at the documentation (such as it is) in the header comment block suggests that you need code in your "libstdc++-dg-prune" procedure to recognize the unsupported cases and return the "::unsupported::freestanding" string.
Digging briefly into the code, the comp_output contents are only checked for those tags (at line 756 in lib/dg.exp on current Git master) if the ${tool}-dg-prune procedure is defined. Do you have such a procedure? If not, try "proc libstdc++-dg-prune { target text } { return text }" instead of your workaround below as the simplest option. I may be able to suggest a better solution after looking at the rest of the testsuite.
However, in combination with dg-warn and dg-error, this does not behave as intended: the errors those expect will not be emitted, and extra errors will be emitted, hence files that should be UNSUPPORTED are counted as FAILs (test for excess errors/test for errors, line NN) as well as UNSUPPORTED.I have a workaround for this issue: diff --git a/libstdc++-v3/testsuite/lib/libstdc++.exp b/libstdc++-v3/ testsuite/lib/libstdc++.exp index f9a165c3d61..526f0fbab73 100644 --- a/libstdc++-v3/testsuite/lib/libstdc++.exp +++ b/libstdc++-v3/testsuite/lib/libstdc++.exp@@ -428,6 +428,10 @@ proc libstdc++-dg-test { prog do_what extra_tool_flags } { set unsupported_message [libstdc++_check_unsupported_p $comp_output]if { $unsupported_message != "" } { + upvar dg-messages messages + upvar dg-excess-errors-flag excess-errors-flag + set messages [list] + set excess-errors-flag 1 set comp_output "::unsupported::$unsupported_message" }... however, it feels prone to breakage (since it depends on how dg-test is implemented).
Where is the Git repository this patch applies to? I will take at least a brief look at the testsuite.
Is there a better way to solve this?
Another option might be to add a target selector to the "dg-do" line that will match all targets except freestanding. Unfortunately there is no simple way to say "except these" in the pattern match rules used here, so a ${tool}-dg-prune procedure is probably your best option, based on what I have seen thus far.
Thanks in advance, and have a most wonderful day,
You are welcome. -- Jacob
[Prev in Thread] | Current Thread | [Next in Thread] |