Re: [Tsp-devel] A test campaign manager for dtest

[Frederik Deweerdt]

On Fri, Apr 11, 2008 at 04:08:45PM +0200, Decherf wrote:
> Hi all,
> 
Hi!
>  
> 
[...]
> 
> The attached file details the main goals of the project. Please let us know
> if you are interested in such an evolution inside dtest. Your potential
> remarks and/or propositions would of course be greatly appreciated.
Nice proposal, thanks!

I've got a few questions/remarks:

- IMHO, this should be part of the dtest distribution
- Would'nt YAML[1] be more flexible than XML? e.g. :

        server_ip: [1.1.1.1]
        client_ip: [2.2.2.2, 3.3.3.3, 4.4.4.4]

        tests:                          # Anything not being 'tests' is
                                        # passed to the tested object as-is ...
        - /home/test/test1:
               server_ip: [5.5.5.5]     # ... unless it is overloaded
        - /home/test/test2:
        - /home/test/test3:
this would lessen the tests's burden of parameter parsing. We could in
particular avoid:
<parameter name="client_ip1" value="2.2.2.2"/>
<parameter name="client_ip2" value="2.2.2.2"/>

- How does the campaign know about a particular test failing? Do we rely
  on forwarding the test's TAP output? In that case, the output needs to be
  serialized it if several clients are running at the same time.

Regards,
Frederik

[1] http://yaml.org/, http://pyyaml.org/wiki/PyYAMLDocumentation