automake
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Automated regression testing‏


From: Jef Driesen
Subject: Automated regression testing‏
Date: Fri, 08 Oct 2010 21:37:02 +0200
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.12) Gecko/20100915 Lightning/1.0pre Thunderbird/3.0.8

Hi,

I have a couple of questions about running tests. I already know how to build and run a typical unit tests, with

check_PROGRAMS = ...
TESTS = ...

But in my case the situation is a little different. I'll start with a little description of the problem I want to solve. I'm writing a library to retrieve data from a number of hardware devices, and parse that data afterwards. Since automated regression testing with real hardware is simply not possible, I wrote a device simulator that serves some data that was previously obtained from a real device.

Now, my questions are:

1. How do I start and stop my device simulator?

In order to run the test application, the device simulator needs to be running. And afterwards it needs to be shutdown again. How can I do this? The device simulator is not a very sophisticated application. It basically runs an infinitive loop, and needs to be aborted with Ctrl-C. So it's not daemonized in any way. How can I do this from the makefiles?

2. How do I run multiple tests with only a single application?

My test application can test several scenario's but only one at a time. To test each possible scenario, the test application has to be run multiple times, but with different options. Or with the same options, but with a different data file loaded in the simulator.

This is different from a typical unit test, where all the tests are performed inside a single test application, so I'm not really sure how do achieve this.

3. How do I detect success/failure based on the output of a test application?

I mainly want to run regression testing on the parsing of the data. The test application writes the parsed data to some file, and then I want to compare the output with some ground thruth data. If they are not identical the test should fail. So it's not just the exit code that I want to capture.

Thanks in advance,

Jef




reply via email to

[Prev in Thread] Current Thread [Next in Thread]