Re: [Discuss-gnuradio] USRP I/O Buffering

[Peter Witkowski]

Not sure I follow.  If I have a large enough buffer, the data coming in and the data coming out should be free of concurrency issues and should be able to work just fine. That is, as long as the producer thread can keep adding data to the message queue, I should be OK.  If it gets locked out due to concurrency issues, I can see that this is where there could be issues (e.g., I drop packets because the producer thread can't push data since it's locked out of the message queue).  Also, a large enough buffer could also alleviate the issue all together since I would be able to record for hours before the overflow would take place.

My sample rates are on the order of 400 MB/s (well within PCIe x4 spec).  Also, my RAID array has been benchmarked to handle roughly 800 MB/s.  The flow-graph is nothing more than a USRP source tied into a File Meta Sink.

On Sun, Sep 7, 2014 at 4:35 PM, Marcus D. Leech <address@hidden> wrote:

On 09/07/2014 04:24 PM, Peter Witkowski wrote:

Hello,

I have a simple application written in Python using GNURadio.  All I am trying to accomplish is to have the USRP data be written to disk.  The application works fine when I dump data to /dev/null or run it at reduced sampling rates.  However, when I run at my desired sampling rate, I have a good amount of buffer overflows (a series of "O" characters get printed).

The host machine that I am working on should have no problems sampling at the higher rates, but I have found a curious issue in that not a whole lot of memory is used up by my GNURadio application.

As a result, I am wondering if there is any way to tell GNURadio to use larger buffers (on the order of a few GB) in order to prevent data from being dropped.  I noted that there seem to be several function calls available in the C++ API, but there seems to be a limited set of these calls in the Python wrappers.  Latency is a non-issue for me at the moment, but I need to capture all the data without dropping a large amount of data.  Note that I have the code running with "real-time" priorities in Linux.

Thanks for your help.  FYI I am running GNURadio on Ubuntu 14.04.  Also, I know that my RAID set-up is capable of writing to disk at twice the rate of data coming in per benchmarking the HDDs.

--
Peter Witkowski
address@hidden

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

Adding buffering for long-term recording simply delays, by a few seconds, that point at which your system cannot keep up.


What sample rates are you trying to record?  What does your flow-graph look like?

Buffering is useful to allow you to "ride through" short-term shortfalls in the ability to handle samples.   It is useless for handling the situation
 where your long-term ability to keep up falls short of what you actually need.

Have you tried setting up a ramdisk, and writing to that?

-- 
Marcus Leech
Principal Investigator
Shirleys Bay Radio Astronomy Consortium
http://www.sbrac.org

--
Peter Witkowski
address@hidden