Re: [Discuss-gnuradio] Virtual memory problem assumably caused by threaded process

[Michael Dickens]

> On Fri, Aug 10, 2007 at 07:41:04PM +0200, Christian Sokolowski wrote:
> >  class _queue_watcher_thread(_threading.Thread):
> >         def __init__(self, rcvd_pktq, callback):
> >                 _threading.Thread.__init__(self)
> >                 self.setDaemon(1)
> >                 self.rcvd_pktq = rcvd_pktq
> >                 self.callback = callback
> >                 self.keep_running = True
> >                 self.start()
> >
> >         def run(self):
> >                 while self.keep_running:
> >                         msg = self.rcvd_pktq.delete_head()
> >                         payload = msg.to_string()
> >
> >                         if self.callback:
> >                                 self.callback(payload, bits_correct, bits_wrong)
> >
> >  I wonder if the _queue_watcher_thread is closed each time at the  
> > end of the
> >  while loop in main properly when I set the object rx=0. Do I need to
> >  consider some memory deallocation issues?

On Aug 13, 2007, at 3:13 AM, Eric Blossom wrote:

> I think that you are right that something is leaking memory, and that
> it could be the _queue_watcher_thread.  You should be able to confirm
> the failure to delete the thread by watching with ps.  The threads
> associated with python will continue to grow.
>
> To get info about threads on GNU/Linux:
>    ps -eLf
>    ps axms
>
>
> Other things to check:
>
>   Is anybody is setting the _queue_watcher_thread self.keep_running  
> to False?
>
>   The loop could be hung in the rcvd_pktq.delete_head().  Somebody may
>   need to to insert_tail a special message that says "exit loop".
>
>
>     def run(self):
>         while self.keep_running:
>             msg = self.rcvd_pktq.delete_head()
>             ok, payload = packet_utils.unmake_packet(msg.to_string 
> (), int(msg.arg1()))
>             if self.callback:
>                 self.callback(ok, payload)

I investigated this earlier this year; I don't know if it caused a  
memory lead or not, but it did cause other issues.  The issue of  
multiple threads makes a difference when running a python script from  
another python script: if the extra thread isn't stopped properly,  
then control won't be returned to the calling script.  The solution  
is for the owning thread to "join" the other thread.  Here is example  
code for this particular application.  You'll need to find a way to  
call "stop()" (or whatever you choose to name this method)  
appropriately since it won't be called by default when doing fg.stop 
().  The "best" solution would be to create a "stop" method in  
"rx_graph" (or a method inheriting from it) which first calls  
"_queue_watcher_thread".stop(), then "inherited_fg".stop().  Note  
that the "_queue_watcher_thread" must be called first, while data is  
still being received, so that the thread doesn't hang waiting for a  
new received packet.

class _queue_watcher_thread(_threading.Thread):
    def __init__(self, rcvd_pktq, callback):
        _threading.Thread.__init__(self)
        self.setDaemon(1)
        self.rcvd_pktq = rcvd_pktq
        self.callback = callback
        self.keep_running = True
        self.start()

    def run (self):
        while (self.keep_running):
                msg = self.rcvd_pktq.delete_head()
                payload = msg.to_string()
                if self.callback:
                        self.callback(payload, bits_correct, bits_wrong)

    def stop (self):
        # tell the thread to not keep running
        self.keep_running = False
        # join the thread to wait for it to finish
        self.join ()

Hope this helps and/or makes sense. - MLD