Re: Using DejaGnu in a regression farm

[Rob Savoye]

Sean Cavanaugh wrote:

> I'm looking at using DejaGnu to help automate a regression testing farm. We 
> plan to have a number of boards connected to the test computer via serial and 
> network, and will be testing the Board Support Packages we're developing 
> (including testing everything from the Linux kernel to userland applications).
>
> > From what I have seen over the last few days of working with it, it seems like 
> > DejaGnu would be a useful tool for this application. However, my experience 
> > with it is limited, so I would like to ask those with more experience:
>
> Has anyone done something like this?  Is it (not) a good idea? If we do go 
> ahead with this, is it something the DejaGnu community would like to see 
> contributed back to the project?

  Actually the original design conditions were to support a testing 
farm. Back at Cygnus we had a room full of embedded boards and weird 
unix machines, and would build and test GCC/GDB/Binutils in an automated 
fashion 24/7. DejaGnu supplied testing support for remote unix machines 
or embedded systems over serial connections. Each SBC had an X10 
controller, and we could automatically reboot the board if a test case 
hung the remote system, which is common when working on fresh compiler 
ports...

  We also used DejaGnu to test libgloss, which was the BSP library for 
GCC/GDB. That ran on many different processors, but that's what DejaGnu 
was designed to handle.

  The higher level code that we used to automate all the testing was 
never part of DejaGnu, and was pretty heavily customized for our own 
particular setup. If worked pretty good, but once we had a few dozen 
systems, maintaining it became close to a full-time job.

  It would be good to add support for this in a more generic fashion. I 
imagine quite a few other people have similar testing farms up and 
running as well. For any application that already has DejaGnu style test 
suites, testing it remotely would be reasonably easy. Otherwise you'd 
need to write your own test suites.

        - rob -