Re: [Discuss-gnuradio] making gnuradio blocks entirely in python

[Josh Blum]

On 09/26/2011 09:22 AM, Johnathan Corgan wrote:
> On Mon, Sep 26, 2011 at 09:05, Josh Blum <address@hidden> wrote:
> 
>> The implementation is relying on a hairy thing like swig directors.
>> Shiver...
> 
> We've had mixed results with swig directors in the past.  The bigger
> issue here, though is the Python global interpreter lock.  This is
> released by the GNU Radio runtime in top_block.start() and
> top_block.run() so that the Python calling thread can proceed after
> GNU Radio kicks off the flowgraph threads.
> 
> When calling back up to Python, that lock has to be re-acquired, and
> all other Python threads in operation (including GUI threads for
> wxPython and QTGui) will block waiting for it to be released.
> 
> So while this would work in theory, it could create a nasty coupling
> in application performance between signal processing code and GUI
> code, and possibly even deadlocks if the non-GNU Radio Python code
> needed to interact with the Python block.
> 
> Fortunately, you've done this in a way that can be tested without any
> modifications to GNU Radio, so we can get some empirical results.
> 
> I think a more useful area for C++->Python communication is to make it
> *easy* for a GNU Radio block to notify a Python object that something
> has happened, possibly with data attached.
> 
> For example, in the uhd_fft.py and other GUI programs that use the fft
> sink, we have to end the GNU Radio flowgraph proper in a message
> sink/message queue, and then have a Python thread run this queue for
> entries, process them, then post the processed FFT frames to
> wxPython's GUI thread.  This is not only inefficient, but hard to
> generalize from or even just learn from by inspection.  I created the
> gru.msgq_runner class to help this, but it's still a hack.
> 
> So a mechanism that takes an arbitrary pmt on the GNU Radio C++ end
> and results in a function call to a Python bare function or class
> method with that pmt as the argument would create a generic method to
> pass events of all sorts up from the "data plane" portion of the app
> to the "control plane", non-GNU Radio code.
> 
>

So a little bit about the implementation I used: The communication
between pyblock_gateway.cc and python is actually message based.

When I made the announcement, the implementation used a python thread to
block on a call, get a message, and dispatch work. This worked, but I
was kind of annoyed dealing with the python thread and getting it to
exit. And its definitely more ideal to have the scheduler call directly
into this python work(). So last night I changed the implementation to
use a SWIG director. So the python handler is called to read the message
and dispatch work.

- In the case of the thread implementation, you need to take the GIL
locking into account. Basically, I did the same thing as the message
queue blocking calls.

- In the case of the director, the GIL locking also has to be taking
into account. So, I grabbed the GIL locker implementation from gr_feval.

So basically, any implementation has to deal with unlocking the GIL
properly. I would like to see how well this current implementation works
for people. I think its the right way to do it, but SWIG directors still
seem like black magic so I am less trustworthy of them.

I will add a throw exception when the default handler gets called (means
something about directors didnt build right on your machine).

_josh