RE: QA Tests: Python vs C++

[Jeff S]

I think I found a good argument for using Python tests vs C++, since speed is 
not really an issue (i.e., typically compile once, and run time differences do 
not seem that important since the tests are not running continuously).

The Python bindings for a block I'm working on were not migrated to v3.9 
correctly, but the C++ test didn't catch it.  I didn't find it until it ran in 
a flow graph in GRC, where it failed.  This (to me) significantly outweighs the 
capability of creating test scenarios to step the debugger through and into the 
code under test.  "C++ tests do not test the Python bindings" is a pretty 
strong statement.

Thanks for the input.
Jeff

-----Original Message-----
From: Discuss-gnuradio <discuss-gnuradio-bounces+e070832=hotmail.com@gnu.org> 
On Behalf Of Marcus Müller
Sent: Monday, March 7, 2022 11:20 AM
To: discuss-gnuradio@gnu.org
Subject: Re: QA Tests: Python vs C++

Hi Jeff,

since all the data handling is done in C++ land in GNU Radio anyways, the only 
difference in test performance is when the Python code takes longer to evaluate 
the the result of some flow graph.

For example:
if you write a test case where the data from a vector sink is compared to the 
data from some reference numpy vector, and you do that with a test function 
that recognizes both reference and test data are iterables, and then, in Python 
code, compares element-by-element, then that's slow.

If you just did

self.assertTrue(numpy.array_equiv(reference, self.sink.data()))

that should be as fast as doing it C++.

Best regards,
Marcus

On 07.03.22 16:40, Josh Morman wrote:
> Jeff,
> 
> When tests are done in C++, they also must be compiled, which adds to 
> the overall gnuradio compilation time.  In-tree the c++ tests are 
> reserved mainly for testing the really low level like buffers.
> I agree with you that an all c++ target makes debugging easy, but you 
> can launch the python flowgraphs with the GDB debugger using program: 
> "/usr/bin/python3" and args:
> /path/to/the/qa_xxx.py.
> 
> Josh
> 
> 
> 
> On Mon, Mar 7, 2022 at 8:37 AM Jeff S <e070832@hotmail.com 
> <mailto:e070832@hotmail.com>>
> wrote:
> 
>     I started writing some QA tests which were missing for some blocks I’m 
> working on (in
>     maint-3.9).  I decided to compare using Python vs using C++ when building 
> new tests. 
>     When I started looking into the C++ tests, it seems that there are not a 
> lot of
>     examples around, so I started wondering why people may stick to 
> Python over C++.____
> 
>     __ __
> 
>     I found Python quicker to code and easier to see what’s being tested, but 
> C++ would
>     run the same test as the Python much quicker (according to the time 
> output from make
>     test).  Writing in C++ also gives me the ability to run Visual Studio 
> Code in debug
>     easier and target sections of code under test, which is a very nice 
> feature.  Visual
>     Studio Code seems to have problems with mixed languages under its 
> visual debugging.____
> 
>     __ __
> 
>     Are there other aspects of Python for QA tests that I’m missing as to why 
> it’s the
>     preferred method?  I’m indifferent as to the tool used because I’ll use 
> whatever gets
>     the job done, so I’m not trying to make this a language pro/con 
> question.____
> 
>     __ __
> 
>     Thanks,____
> 
>     Jeff____
>