dejagnu
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Excluding FAILs from UNSUPPORTED test cases


From: Jacob Bachmeyer
Subject: Re: Excluding FAILs from UNSUPPORTED test cases
Date: Fri, 23 Sep 2022 23:24:56 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.22) Gecko/20090807 SeaMonkey/1.1.17 Mnenhy/0.7.6.0

Jonathan Wakely wrote:
[...]
Yes, this is a little surprising. If a test is UNSUPPORTED because it
fails to match a target selector, then any dg-error or other
directives in the test are ignored, and there are no PASS, XFAIL etc.
lines for them in the results.

If a target selector fails to match, the test is not run at all (line 683 in current dg.exp) and the ${tool}-dg-test and ${tool}-dg-prune procedures are never invoked.

But when it's unsupported because ${tool}-dg-prune returns
"::unsupported::" the individual checks within the file still appear
in the results. I realise that in the target selector case, the test
is just skipped before doing anything, and in the prune case we don't
know that it's unsupported until *after* testing it. But it seems to
me that ideally the individual checks would get "retroactively
skipped" if tool-dg-prune returns one of ::untested::, ::unresolved::,
or ::unsupported::. Maybe that's not easy to do though.

This actually looks like a bug in dg.exp to me. The output from ${tool}-dg-test is first collected, all at once, (line 696) and then analyzed (lines 701-749). I would expect ${tool}-dg-prune to have an opportunity to modify the text to be analyzed /before/ the analysis is done, but the current code checks for messages first, /then/ calls ${tool}-dg-prune (and the framework's prune_warnings procedure) to remove irrelevant output, then declares excess errors if the final text is nonempty.

To Johnathan and Arsen: if you agree that this seems like a bug, please file a report at <bug-dejagnu@gnu.org> to assign a PR number to the issue. If there is no good reason for the current behavior, a bug known to actually bite GCC is a fairly solid reason to change the code.

To Rob: is there a good reason known for the current behavior from the times of old? The code in question goes back to the "Initial revision" in the current repository and a quick search of the ChangeLogs sheds no light on the matter.


-- Jacob




reply via email to

[Prev in Thread] Current Thread [Next in Thread]