dejagnu
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Excluding FAILs from UNSUPPORTED test cases


From: Jacob Bachmeyer
Subject: Re: Excluding FAILs from UNSUPPORTED test cases
Date: Thu, 22 Sep 2022 22:33:42 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.22) Gecko/20090807 SeaMonkey/1.1.17 Mnenhy/0.7.6.0

Arsen Arsenović wrote:
Hi,

While working on freestanding libstdc++, we came up with the idea to use a special error message to detect tests that use headers that don't work in freestanding, and mark them as unsupported by emitting ::unsupported:: from libstdc++-dg-test.

I have not yet had the occasion to dive into the dg.exp code, but a quick look at the documentation (such as it is) in the header comment block suggests that you need code in your "libstdc++-dg-prune" procedure to recognize the unsupported cases and return the "::unsupported::freestanding" string.

Digging briefly into the code, the comp_output contents are only checked for those tags (at line 756 in lib/dg.exp on current Git master) if the ${tool}-dg-prune procedure is defined. Do you have such a procedure? If not, try "proc libstdc++-dg-prune { target text } { return text }" instead of your workaround below as the simplest option. I may be able to suggest a better solution after looking at the rest of the testsuite.

However, in combination with dg-warn and dg-error, this does not behave as intended: the errors those expect will not be emitted, and extra errors will be emitted, hence files that should be UNSUPPORTED are counted as FAILs (test for excess errors/test for errors, line NN) as well as UNSUPPORTED.

I have a workaround for this issue:
diff --git a/libstdc++-v3/testsuite/lib/libstdc++.exp b/libstdc++-v3/
testsuite/lib/libstdc++.exp
index f9a165c3d61..526f0fbab73 100644
--- a/libstdc++-v3/testsuite/lib/libstdc++.exp
+++ b/libstdc++-v3/testsuite/lib/libstdc++.exp
@@ -428,6 +428,10 @@ proc libstdc++-dg-test { prog do_what extra_tool_flags } { set unsupported_message [libstdc++_check_unsupported_p $comp_output]
     if { $unsupported_message != "" } {
+       upvar dg-messages messages
+       upvar dg-excess-errors-flag excess-errors-flag
+       set messages [list]
+       set excess-errors-flag 1
        set comp_output "::unsupported::$unsupported_message"
     }
... however, it feels prone to breakage (since it depends on how dg-test is implemented).

Where is the Git repository this patch applies to? I will take at least a brief look at the testsuite.

Is there a better way to solve this?

Another option might be to add a target selector to the "dg-do" line that will match all targets except freestanding. Unfortunately there is no simple way to say "except these" in the pattern match rules used here, so a ${tool}-dg-prune procedure is probably your best option, based on what I have seen thus far.

Thanks in advance, and have a most wonderful day,
You are welcome.


-- Jacob




reply via email to

[Prev in Thread] Current Thread [Next in Thread]