[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Automated regression testing
From: |
Jef Driesen |
Subject: |
Automated regression testing |
Date: |
Fri, 08 Oct 2010 21:37:02 +0200 |
User-agent: |
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.12) Gecko/20100915 Lightning/1.0pre Thunderbird/3.0.8 |
Hi,
I have a couple of questions about running tests. I already know how to
build and run a typical unit tests, with
check_PROGRAMS = ...
TESTS = ...
But in my case the situation is a little different. I'll start with a
little description of the problem I want to solve. I'm writing a library
to retrieve data from a number of hardware devices, and parse that data
afterwards. Since automated regression testing with real hardware is
simply not possible, I wrote a device simulator that serves some data
that was previously obtained from a real device.
Now, my questions are:
1. How do I start and stop my device simulator?
In order to run the test application, the device simulator needs to be
running. And afterwards it needs to be shutdown again. How can I do
this? The device simulator is not a very sophisticated application. It
basically runs an infinitive loop, and needs to be aborted with Ctrl-C.
So it's not daemonized in any way. How can I do this from the makefiles?
2. How do I run multiple tests with only a single application?
My test application can test several scenario's but only one at a time.
To test each possible scenario, the test application has to be run
multiple times, but with different options. Or with the same options,
but with a different data file loaded in the simulator.
This is different from a typical unit test, where all the tests are
performed inside a single test application, so I'm not really sure how
do achieve this.
3. How do I detect success/failure based on the output of a test
application?
I mainly want to run regression testing on the parsing of the data. The
test application writes the parsed data to some file, and then I want to
compare the output with some ground thruth data. If they are not
identical the test should fail. So it's not just the exit code that I
want to capture.
Thanks in advance,
Jef
- Automated regression testing,
Jef Driesen <=