Re: [Tsp-devel] A document generator for DTest

[Eric Noulard]

2008/4/16, T. Decherf <address@hidden>:
> Hi,
>
>  Didier Barvaux and me have the need of generating test reports when using
> DTest. We propose here a possible implementation of a document generator
> which is totally independant. However it would probably be convenient to
> have such functionalities integrated inside DTest.
>  The attached document details the main goals of the project. Please let us
> know if you are interested in such an evolution in DTest.

Yes personnally I am, however I have some remark.

>  Remarks and propositions are of course most welcome.

I would rather rename your
"DocumentGenerator" class to "TraceHandler" class
and the "DocumentManager" to "TraceManager".

In fact I think the "document report" of the test sequence should be seen as
a particular trace generated from the sequence itself.

And in fact if I understand your objectives well the
TraceManager  may either be "merged" with
"DTestMaster" itself or became a "member".
Then each DTester will call DTestMaster.traceManager.xxxxx()
for each step they are currently running.

I think DocumentGenerator/DocumentManager follow
the same pattern as DTester/DTestMaster.

I would modify DocumentGenerator/TraceHandler with the following:

0) add addDTester : which will indicate the trace handler
    that the specified dtester is involved in the distributed test
    sequence.

1) add initialize() method which may be used to build
    the header of the report which may indicates
    the list of dtesters involved in the test (with their name and location)
    the number of planned steps.

2)  I think save(String) should not be visible since finalize() is fine.
    You may
        docs.registerDocument(HTMLGenerator("/path/to/file"))
    then
        docs.finalize() will do the ,job.

    I think it's better since if you registered several "file oriented"
    traceHandler the following
        docs.save("/path/file_name")
     will need some per-handler name mangling scheme in order
     to handle the latex, HTML, TAP, ... several file name need.

Concerning

AppendTest
and
AppendTestStep

I would rather go for
 testTitle
and
 testStepResult

When rereading your proposal as a whole I would say that I may have missed
something important.

Do you want to be able to generate:
 A) one report for a single DTest sequence
 B) one report for SEVERAL DTest sequences?

I had A) in mind but I now realize that B) may be more useful.
I you target is B) then an autonomous
TraceManager class is definitely needed.

moreover I would rather replace testTitle with

addTestSequence(testTitle,testSummary,dtestmaster1)

the DTestMaster (dtestmaster1) passed as parameter represent
a "single" dtest sequence.
then initialize() and finalize() are not enough.
we need

initializeSequence() and finalizeSequence()
which will do what is needed for each dtest sequence
i.e. each DTestMaster.startTestSequence()/myDTestMaster.waitTestSequenceEnd()

Here comes a possible sequence:

traceManager = dtest.TraceManager()
traceManager.register(HTMLTraceHandler("myreport.html")
traceManager.register(LatexTraceHandler("myreport.tex")

<DTester tester1, tester2 defines their sequence>
...
dtestmaster1 = dtest.DTestMaster()
dtestmaster1.register(tester1)
dtestmaster1.register(tester2)
traceManager.addTestSequence("Test 1", "A complex distributed test",
dtestmaster1)
dtestmaster1.startTestSequence()
dtestmaster1.waitTestSequenceEnd()

<DTester tester3, tester4 defines their sequence>
...
dtestmaster2 = dtest.DTestMaster()
dtestmaster2.register(tester3)
dtestmaster2.register(tester4)
traceManager.addTestSequence("Test 2", "A really complex distributed
test",dtestmaster2)
dtestmaster2.startTestSequence()
dtestmaster2.waitTestSequenceEnd()
...

# here we "finalize" every registered TraceHandler.
traceManager.finalize()

using this scheme each DTestMaster used in a "addTestSequence"
will call TraceManager.initializeSequence() when
DTestMaster.startTestSequence()
is called (same for TraceManager.finalizeSequence()/waitTestSequenceEnd())

during the sequence each DTester registered in the DTestMaster will
call the DTestMaster.traceManager.testStepResult() with appropriate
arguments build from ANY step (ok, barrier, etc...)

DTestMaster may have a "default" TraceManager built-in which includes
a TAPTraceHandler and/or nothing at all.

in fact we may reverse the call to:
traceManager.addTestSequence("Test 2", "A really complex distributed
test",dtestmaster2)

into:

dtestmaster2.registerTraceManager(traceManager,"Test 2", "A really
complex distributed test")

My mail is already too long but I think I've layed out my ideas :=)
I'm waiting for you remark too.

May be next time we may exchange python code for
dtest.TraceManager
and
dtest.TraceHandler (an interface-like one would be enough for specs)


-- 
Erk