[Octave-bug-tracker] [bug #55522] Add failure summary to __run_test_suite__ output?

[Andrew Janke]

URL:
  <https://savannah.gnu.org/bugs/?55522>

                 Summary: Add failure summary to __run_test_suite__ output?
                 Project: GNU Octave
            Submitted by: apjanke
            Submitted on: Sun 20 Jan 2019 05:58:40 AM UTC
                Category: Test Suite
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Feature Request
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
         Discussion Lock: Any
                 Release: dev
        Operating System: Any

    _______________________________________________________

Details:

At the end of a run, __run_test_suite__ will output the number of failures,
but it won't list which files had tests that failed, so you have to scroll
back an manually scan through its long output.

How about adding a list of the files with failed tests to the results summary,
so it's easier to identify where you need to go looking for problems?

Here's a patch to do so.

After:


>> __run_test_suite__
[...]
  fixed/system.tst ............................................... PASS 
103/103
  fixed/transpose.tst ............................................ PASS  
16/16
  fixed/try.tst .................................................. PASS  
18/18
  fixed/unwind.tst ............................................... PASS   
2/2
  fixed/while.tst ................................................ PASS   
6/6

Summary:

  PASS                            14948
  FAIL                               17
  XFAIL (reported bug)               54
  SKIP (missing feature)             50
  SKIP (run-time condition)          29

  Failed tests:
    
/Applications/Octave-4.4.1.app/Contents/Resources/usr/Cellar/octave-octave-app_4.4.1_0/4.4.1/share/octave/4.4.1/etc/tests/libinterp/corefcn/help.cc-tst
    
/Applications/Octave-4.4.1.app/Contents/Resources/usr/Cellar/octave-octave-app_4.4.1_0/4.4.1/share/octave/4.4.1/etc/tests/libinterp/corefcn/pr-output.cc-tst
    
/Applications/Octave-4.4.1.app/Contents/Resources/usr/Cellar/octave-octave-app_4.4.1_0/4.4.1/share/octave/4.4.1/etc/tests/libinterp/corefcn/variables.cc-tst
    
/Applications/Octave-4.4.1.app/Contents/Resources/usr/Cellar/octave-octave-app_4.4.1_0/4.4.1/share/octave/4.4.1/etc/tests/libinterp/parse-tree/oct-parse.in.yy-tst
    
/Applications/Octave-4.4.1.app/Contents/Resources/usr/Cellar/octave-octave-app_4.4.1_0/4.4.1/share/octave/4.4.1/m/image/getframe.m
[...]




    _______________________________________________________

File Attachments:


-------------------------------------------------------
Date: Sun 20 Jan 2019 05:58:40 AM UTC  Name:
test_suite_add_failure_summary_to_results_output.patch  Size: 6KiB   By:
apjanke

<http://savannah.gnu.org/bugs/download.php?file_id=46034>

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?55522>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/