dejagnu
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[patch] handle 'variations' when doing XML output.


From: cgd
Subject: [patch] handle 'variations' when doing XML output.
Date: 30 Apr 2003 15:26:23 -0700

So, i was messing around with dejagnu, testing GCC with multiple
multilib variations, e.g.:

gmake check-gcc RUNTESTFLAGS="--target_board 
'remote4-cfe{-mips1,-mips2,-mips3,-mips4}' -x execute.exp=20020227-1.c"

so that i could test -mips1, -mips2, -mips3, and -mips4 multilibs.

The log file is nice enough to say:

> Schedule of variations:
>     remote4-cfe/-mips1
>     remote4-cfe/-mips2
>     remote4-cfe/-mips3
>     remote4-cfe/-mips4

for the example above.


I notice that when doing this and generating XML output, dejagnu
outputs a structure like:

testsuite
  test          # tests and summaries for variation 1
  test
  test
  summary
  summary
  test          # tests and summaries for variation 2
  test
  test
  summary
  summary
  ...
  summary       # summary totals
  summary


It doesn't even provide any indication of which tests are for which
variations, etc.!  (there's not even any indication of variations
tested, in the XML output!)

The patch below, which has been tested by running the tests mentioned
above on current GCC sources (from sources.redhat.com dejagnu sources)
and sanity checked by hand and with xmllint, produces a structure like

testsuite
  variation
    name
    test
    test
    test
    summary
    summary
  variation
    name
    test
    test
    test
    summary
    summary
  ...
  summary
  summary

if testing multiple variations.

For compatibility with peoples' existing tools, it leaves the
single-variation test output the same.  Really, i don't think that's
"the right thing," but better to avoid breaking peoples' code, IMO.
I'd be glad to adjust the XML output to always put tests under
'variations', which is IMO the right thing, if people would prefer
that.  8-)

FWIW, I'm also not convinced that the current XML output behaviour of
outputting only the summary stats which are non-zero is really the
right thing.  IMO, better to output all of them all the time (so that
comparison programs don't necessarily have to know the complete set of
stats one might have, to compare/print nicely).  But i don't care so
much about that.  I'd be willing to create the patch for that, too, if
people wanted it.


cgd
--
2003-03-29  Chris Demetriou  <address@hidden>

        * runtest.exp: Invoke begin_variation and end_variation
        when testing multiple variations.
        * lib/framework.exp (insertdtd): Update DTD for variation
        notations.
        (begin_variation, end_variation): New functions.

Index: runtest.exp
===================================================================
RCS file: /cvs/src/src/dejagnu/runtest.exp,v
retrieving revision 1.11
diff -u -p -r1.11 runtest.exp
--- runtest.exp 14 Sep 2002 00:31:38 -0000      1.11
+++ runtest.exp 30 Apr 2003 22:15:43 -0000
@@ -1630,6 +1630,9 @@ foreach current_target $target_list {
     # Set the counts for this target to 0.
     reset_vars;
     clone_output "Running target $current_target_name"
+    if { $target_count > 1 } {
+       begin_variation;
+    }
 
     setup_target_hook $current_target_name $current_target;
 
@@ -1903,7 +1906,7 @@ foreach current_target $target_list {
     }
     cleanup_target_hook $current_target;
     if { $target_count > 1 } {
-       log_summary;
+       end_variation;
     }
 }
 
Index: lib/framework.exp
===================================================================
RCS file: /cvs/src/src/dejagnu/lib/framework.exp,v
retrieving revision 1.9
diff -u -p -r1.9 framework.exp
--- lib/framework.exp   14 Sep 2002 00:31:38 -0000      1.9
+++ lib/framework.exp   30 Apr 2003 22:15:45 -0000
@@ -47,13 +47,14 @@ proc check_xml { } {
 proc insertdtd { } {
     xml_output "<!DOCTYPE testsuite \[
 <!-- testsuite.dtd -->
-<!ELEMENT testsuite (test | summary)+>
+<!ELEMENT testsuite (variation | test | summary)+>
+<!ELEMENT variation (name | test | summary)+>
+  <!ELEMENT name               (#PCDATA)>
 <!ELEMENT test (log, result, name, prms_id )>
-  <!ELEMENT log                        (#PCDATA)>
+  <!ELEMENT log                (#PCDATA)>
   <!ELEMENT result             (#PCDATA)>
-  <!ELEMENT name               (#PCDATA)>
   <!ELEMENT prms_id            (#PCDATA)>
-  <!ELEMENT summary (result, description, total)>
+<!ELEMENT summary (result, description, total)>
   <!ELEMENT description        (#PCDATA)>
   <!ELEMENT total              (#PCDATA)>
 \]>"
@@ -107,6 +108,33 @@ proc close_logs { } {
 }
 
 #
+# Do any special log setup needed when starting a new test target
+# (a.k.a. variation) when testing multiple variations.
+#
+proc begin_variation { } {
+    global current_target_name
+    global xml
+
+    if { $xml } {
+        xml_output "  <variation>"
+        xml_output "    <name>$current_target_name</name>"
+    }
+}
+
+#
+# Do any special log cleanup needed when finishing a test target
+# (a.k.a. variation) when testing multiple variations.
+#
+proc end_variation {} {
+    global xml
+
+    log_summary
+    if { $xml } {
+        xml_output "  </variation>"
+    }
+}
+
+#
 # Check build host triplet for pattern
 #
 # With no arguments it returns the triplet string.





reply via email to

[Prev in Thread] Current Thread [Next in Thread]