Re: [Tsp-devel] A test campaign manager for dtest

[Eric Noulard]

2008/4/11, Frederik Deweerdt <address@hidden>:
>  > The attached file details the main goals of the project. Please let us know
>  > if you are interested in such an evolution inside dtest. Your potential
>  > remarks and/or propositions would of course be greatly appreciated.
>
> Nice proposal, thanks!

+1

>
>  I've got a few questions/remarks:
>
>  - IMHO, this should be part of the dtest distribution

I agree to but:

I want to keep DTest as simple as possible.

For example using DTest does not imply using a Test Campagn Manager
DTest should be nicely layered such that lower blocks
may be re-used.

>  - Would'nt YAML[1] be more flexible than XML? e.g. :
>
[...]

>  this would lessen the tests's burden of parameter parsing. We could in
>  particular avoid:
>  <parameter name="client_ip1" value="2.2.2.2"/>
>  <parameter name="client_ip2" value="2.2.2.2"/>

I've no precise idea about that [yet]

>  - How does the campaign know about a particular test failing? Do we rely
>   on forwarding the test's TAP output?

We discussed the fact that "DTest report" may be of different kind
TAP was the first, Lionel already added MSC-like output.

Independently of the "output format" each DTester should (generically)
elaborate a test result. If an ok step has failed then DTester has failed
then DTestmaster know it then Campaign Manager too.

> In that case, the output needs to be
>   serialized it if several clients are running at the same time.

Unless I miss your point I think DTestmaster already serialize
the "distributed test" output?

In fact there is no "real" serialize need since DTestmaster and
all DTester runs on the same machine and are LOCALLY synchronized.
However DTester have asynchronous "output" from the SSH Session Handler.

The tests sequence itself may/should contains expect/barrier steps
which ensure synchronization when (functionnalmy) needed.

I will be silent for a week, since I may not touch a keyboard during
this time :=)


-- 
Erk