discuss-gnuradio
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CSV file as input


From: Marcus Müller
Subject: Re: CSV file as input
Date: Fri, 18 Mar 2022 19:32:20 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.2

Ah cool! Thanks for clarifying :) This sounds to be a rather nice setup, 
analog-wise!

Yeah, then just dumping the raw 32bit unsigned to SD Card is probably easiest.

(by the way, this is << 1Mb/s, so just dumping the raw data over a UART or SPI interface to some serial-to-USB converter might work as well to get the data into your PC. If your ARM does have USB2 built-in, then that would also be a rather cool thing, but knowing the varying quality of chip vendor USB hardware abstractions, that might or might not be easy to implement :) In both cases, UART/SPI serial output converted to USB, or native USB, you'd probably have to afterwards write a schmall C/C++ driver, so that SoapySDR or GNU Radio directly can talk to it.)

Cheers,
Marcus

On 18.03.22 19:26, david vanhorn wrote:
I'm using a PCB that I designed with an ARM chip, codec, and SD card for logging, as my data capture platform. Feeding that is a QSD (Tayloe) front end that I designed, specifically for the 630m ham band, converting down to 1kHz differential I and Q signals to the codec, which has a 105dB SNR. The front end appears to have a 90dB linear dynamic range so far as I can measure with my equipment. I'll improve that if I can. Once I capture to SD, then I can pull the SD and process on the PC to develop weak signal detection.



On Fri, Mar 18, 2022 at 12:12 PM Marcus Müller <mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>> wrote:

    Hey :)

    CSV might or might not be convenient, but if C or assembler is your tool: 
The things that
    the GNU Radio file source reads or the file sink writes is exactly what you 
get when you
    take a buffer of samples and do an `fwrite` on that :) Just a dump of the 
raw memory to a
    file. 32 bit unsigned should be directly digestible by GNU Radio (even if 
there were
    endianness issues – you can just read as bytes and reshuffle as needed :)).

    I didn't fully get how you're currently interfacing your hardware. Care to 
explain in a
    bit more breadth? What are the components of your system, and how does the 
computer
    running GNU Radio relate?

    Best and slightly excited regards,
    Marcus

    On 18.03.22 18:37, david vanhorn wrote:
     > Hi!
     >
     > I'm trying to interface some radio hardware I built to GnuRadio by way 
of data
    captured to
     > SD cards.
     > I have two channels (I and Q) of 32 bit unsigned data internally, and I 
originally
    assumed
     > CSV would be the easy path, but now I see it's not.
     > Coming in through the PC sound card is not an option for me, I'm using a 
particular
    codec
     > selected for the application, and my goal is to develop signal processing
    algorithms to
     > then be implemented back on my processor in C or ASM.
     >
     > I suppose it would be easiest if I rework my hardware to log data as if 
it were the
     > "Signal Source" block with complex output.
     > Where can I see what that looks like at the level of raw data?
     >
     >
     >
     >
     > On Fri, Mar 18, 2022 at 4:59 AM Marcus Müller <mmueller@gnuradio.org
    <mailto:mmueller@gnuradio.org>
     > <mailto:mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>>> wrote:
     >
     >     Hi David,
     >
     >     you could write a quick python block that just reads values from the 
CSV file
    and outputs
     >     them. That'd be a very nice, basic exercise, and I think our freshly 
overhauled
     >     tutorials[1] should bring you there very quickly!
     >     If you want help with that, hit us up in this mailing list (ideally 
after
    reading the
     >     tutorials up to the point of roughly understanding how to write 
(embedded) Python
     >     blocks),
     >     and tell us more about the data in your CSV files.
     >
     >     Alternatively, you could also write a converter of CSV to a format 
that GNU
    Radio by
     >     itself already has a reader for – and the main candidate here would 
probably
    just be
     >     plain
     >     raw data files (as e.g. numpy's `ndarray.tofile("filename")` does) – 
the File
    Source
     >     could
     >     directly read that. But with our freshly rewrite Wavfile sink and 
source
    blocks, we can
     >     write and read most audio files, just as well.
     >
     >     Then your flow graph could do the signal processing you want – e.g 
frequency
    translation,
     >     low-pass filtering… and finally output it to any device that you 
have a GNU Radio
     >     interface to (e.g., your sound card). The hardware runs at a sample 
rate – GNU
    Radio
     >     itself just tries to feed it as fast as possible. So, the signal 
processing in
    GNU Radio
     >     itself isn't concerned with rate at all!
     >
     >     Hope this helps,
     >     Marcus
     >
     >     [1] https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>
    <https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>>
     >
     >     PS: you'll often find me online, recommending not to use CSV as a 
sample
    storage format.
     >     I'll do the same to you here, but not because I think it's in any 
way invalid
    to have
     >     data
     >     in CSV files; I just want to point out it might be worth thinking 
about using
    something
     >     else. So take this with a "I think it's pretty cool you're doing 
this!".
     >
     >     That has the reasons that
     >     a) unless you're more restricted than "CSV" says, you don't know how 
many bits
    are there
     >     per sample, as numbers might be represented in different lengths, so 
seeking
    exactly only
     >     works by reading and understanding the whole file up to the point 
you seek to,
     >     b) conversion of floating point numbers to human-readable form 
incurs rounding
    errors,
     >     and
     >     that can really wreck your day if you need to rerun *exactly* the 
same
    experiment twice,
     >     c) printing numbers as text is really inefficient, both storage-wise 
as well as
    compute
     >     wise (which will only matter at higher sampling rates) and 
sometimes, but only
    sometimes,
     >     ( d) people say that CSV is good because it's human-readable, but I 
challenge
    anyone to
     >     read a text file with only 10000 values and be happier about that 
than if he
    used a tool
     >     that displayed the values graphically, zoomably, and then allows for 
inspection
    of single
     >     values once zoomed sufficiently in.)
     >
     >
     >     On 18.03.22 04:55, david vanhorn wrote:
     >      > I've done a little with Gnuradio a couple years ago, but I'd now 
like to
    apply it to a
     >      > serious problem.
     >      >
     >      > I have a design I'm working on that will output raw data that 
could be
    interpreted
     >     as an
     >      > audio stream centered on 1kHz.  I'd like to work on extracting CW 
signals
    that are
     >     rather
     >      > slow, from a rather narrow bandwidth, and see how far down into 
the noise I can
     >     actually
     >      > extract the signals.
     >      >
     >      > Is there a block that can bring in CSV data from a file at a 
specific rate, and
     >     serve as
     >      > the input to my CW detection system?
     >      >
     >      >
     >      > --
     >      > K1FZY (WA4TPW) SK  9/29/37-4/13/15
     >
     >
     >
     > --
     > K1FZY (WA4TPW) SK  9/29/37-4/13/15



--
K1FZY (WA4TPW) SK  9/29/37-4/13/15



reply via email to

[Prev in Thread] Current Thread [Next in Thread]