Re: Removing XFAILs from test suite summary

[John W. Eaton]

On 04/11/2018 01:54 PM, Rik wrote:
> When I run the test suite, I currently get
>
> Summary:
>
>    PASS                            14855
>    FAIL                                0
>    XFAIL (reported bug)               31
>    XFAIL (expected failure)            5
>    SKIP (missing feature)            138
>    SKIP (run-time condition)          12
>
> There are 5 reported XFAILs.  XFAILs are supposed to be known failures,
> i.e., tests that the programmer has put in temporarily while writing code
> and which are expected to pass as soon as the code is actually viable.  It
> really is more of a development feature, and I don't think we need to ship
> code that has known failures like this.
>
> My suggestion would be that we file a bug report for each remaining XFAIL
> and then associate that bug report with the BIST tests.  This would at
> least give us a place to track these things.  As it is, nobody ever really
> reviews these.  At least with a bug report filed on Savannah there would be
> a record of it.

Yes, I agree that all XFAILS should have bug report numbers associated 
with them.

Is it possible that those five already have bug reports but the numbers 
were never recorded?  If not, then yes, we should create reports for them.

jwe