Re: Excluding FAILs from UNSUPPORTED test cases

[Jonathan Wakely]

On Fri, 23 Sept 2022 at 04:34, Jacob Bachmeyer wrote:
>
> Arsen Arsenović wrote:
> > Hi,
> >
> > While working on freestanding libstdc++, we came up with the idea to use
> > a special error message to detect tests that use headers that don't work
> > in freestanding, and mark them as unsupported by emitting
> > ::unsupported:: from libstdc++-dg-test.
> >
>
> I have not yet had the occasion to dive into the dg.exp code, but a
> quick look at the documentation (such as it is) in the header comment
> block suggests that you need code in your  "libstdc++-dg-prune"
> procedure to recognize the unsupported cases and return the
> "::unsupported::freestanding" string.
>
> Digging briefly into the code, the comp_output contents are only checked
> for those tags (at line 756 in lib/dg.exp on current Git master) if the
> ${tool}-dg-prune procedure is defined.  Do you have such a procedure?

Yes:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libstdc%2B%2B-v3/testsuite/lib/prune.exp;h=70ff302288ed4fc04755168ee2f3fa332186dd8d;hb=HEAD#l31

And indeed, returning ::sunupported::message from there works
perfectly, as all the earlier FAIL results are replaced by that
string. Thanks!

Arsen, see you back on IRC, I have an alternative to your
target-utils.exp changes that fixes it.

> Where is the Git repository this patch applies to?  I will take at least
> a brief look at the testsuite.

The top level file is:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libstdc%2B%2B-v3/testsuite/libstdc%2B%2B-dg/conformance.exp;h=366a22e9a3e6ef5f8169a6d6b6e0cedc2b9c708c;hb=HEAD
Which includes other files from:
https://gcc.gnu.org/git/?p=gcc.git;a=tree;f=libstdc%2B%2B-v3/testsuite/lib;h=8355315188041a63d22bc53122ca1cbb30e7a1a2;hb=HEAD


> > Is there a better way to solve this?
> >
>
> Another option might be to add a target selector to the "dg-do" line
> that will match all targets except freestanding.

Yes, we do already have such an effective target, but the idea was to
avoid having to add the selector to 2000+ files. What we want is that
any test which exits with a specific error will get UNSUPPORTED
automatically, without having to modify the test itself.

And by doing it in libstdc++-dg-prune that works perfectly, thanks again.