Re: CSV file as input

[david vanhorn]

Hi! 

I'm trying to interface some radio hardware I built to GnuRadio by way of data captured to SD cards.

I have two channels (I and Q) of 32 bit unsigned data internally, and I originally assumed CSV would be the easy path, but now I see it's not.

Coming in through the PC sound card is not an option for me, I'm using a particular codec selected for the application, and my goal is to develop signal processing algorithms to then be implemented back on my processor in C or ASM.

I suppose it would be easiest if I rework my hardware to log data as if it were the "Signal Source" block with complex output.

Where can I see what that looks like at the level of raw data?

On Fri, Mar 18, 2022 at 4:59 AM Marcus Müller <mmueller@gnuradio.org> wrote:

Hi David,

you could write a quick python block that just reads values from the CSV file and outputs
them. That'd be a very nice, basic exercise, and I think our freshly overhauled
tutorials[1] should bring you there very quickly!
If you want help with that, hit us up in this mailing list (ideally after reading the
tutorials up to the point of roughly understanding how to write (embedded) Python blocks),
and tell us more about the data in your CSV files.

Alternatively, you could also write a converter of CSV to a format that GNU Radio by
itself already has a reader for – and the main candidate here would probably just be plain
raw data files (as e.g. numpy's `ndarray.tofile("filename")` does) – the File Source could
directly read that. But with our freshly rewrite Wavfile sink and source blocks, we can
write and read most audio files, just as well.

Then your flow graph could do the signal processing you want – e.g frequency translation,
low-pass filtering… and finally output it to any device that you have a GNU Radio
interface to (e.g., your sound card). The hardware runs at a sample rate – GNU Radio
itself just tries to feed it as fast as possible. So, the signal processing in GNU Radio
itself isn't concerned with rate at all!

Hope this helps,
Marcus

[1] https://tutorials.gnuradio.org

PS: you'll often find me online, recommending not to use CSV as a sample storage format.
I'll do the same to you here, but not because I think it's in any way invalid to have data
in CSV files; I just want to point out it might be worth thinking about using something
else. So take this with a "I think it's pretty cool you're doing this!".

That has the reasons that
a) unless you're more restricted than "CSV" says, you don't know how many bits are there
per sample, as numbers might be represented in different lengths, so seeking exactly only
works by reading and understanding the whole file up to the point you seek to,
b) conversion of floating point numbers to human-readable form incurs rounding errors, and
that can really wreck your day if you need to rerun *exactly* the same experiment twice,
c) printing numbers as text is really inefficient, both storage-wise as well as compute
wise (which will only matter at higher sampling rates) and sometimes, but only sometimes,
( d) people say that CSV is good because it's human-readable, but I challenge anyone to
read a text file with only 10000 values and be happier about that than if he used a tool
that displayed the values graphically, zoomably, and then allows for inspection of single
values once zoomed sufficiently in.)


On 18.03.22 04:55, david vanhorn wrote:
> I've done a little with Gnuradio a couple years ago, but I'd now like to apply it to a
> serious problem.
>
> I have a design I'm working on that will output raw data that could be interpreted as an
> audio stream centered on 1kHz.  I'd like to work on extracting CW signals that are rather
> slow, from a rather narrow bandwidth, and see how far down into the noise I can actually
> extract the signals.
>
> Is there a block that can bring in CSV data from a file at a specific rate, and serve as
> the input to my CW detection system?
>
>
> --
> K1FZY (WA4TPW) SK  9/29/37-4/13/15

--

K1FZY (WA4TPW) SK  9/29/37-4/13/15