discuss-gnuradio
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CSV file as input


From: Marcus Müller
Subject: Re: CSV file as input
Date: Fri, 18 Mar 2022 21:25:29 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.2

Like the reproducibility aspect of going for storage, but it means no live signal observation :)

Just for future hardware ideas: with these bitrates, you should be well in range of what the cheaper TOLSLINK optical transmitter modules [1] and receivers [2] could do.

[1] https://www.digikey.com/en/products/detail/everlight-electronics-co-ltd/PLT237-T10WH/14641724 , https://www.tme.eu/en/details/fcr6842031t/optical-connectors/cliff/otj-1-fcr6842031t/
[2] 
https://www.tme.eu/en/details/fcr6842032r/optical-connectors/cliff/orj-3-fcr6842032r/

On 18.03.22 19:53, Marcus D. Leech wrote:
On 2022-03-18 14:48, david vanhorn wrote:
Noise is always an issue.  I could do a serial port over USB, or TTL USART, but I thought that the SD card would be the most quiet, not requiring any electrical connection to the PC.
It also means that I automatically have my recordings available for regression 
testing.

Yeah, I thought that your architecture was probably driven by noise concerns--630M would not be a "forgiving" band in this regard.  I will point out, just as an FYI,   that USB-over-fiber extenders do exist, but because they're rather "niche", they're not cheap at all....



On Fri, Mar 18, 2022 at 12:32 PM Marcus Müller <mmueller@gnuradio.org> wrote:

    Ah cool! Thanks for clarifying :) This sounds to be a rather nice setup, 
analog-wise!

    Yeah, then just dumping the raw 32bit unsigned to SD Card is probably 
easiest.

    (by the way, this is << 1Mb/s, so just dumping the raw data over a UART or 
SPI
    interface
    to some serial-to-USB converter might work as well to get the data into 
your PC. If
    your
    ARM does have USB2 built-in, then that would also be a rather cool thing, 
but
    knowing the
    varying quality of chip vendor USB hardware abstractions, that might or 
might not be
    easy
    to implement :) In both cases, UART/SPI serial output converted to USB, or 
native USB,
    you'd probably have to afterwards write a schmall C/C++ driver, so that 
SoapySDR or GNU
    Radio directly can talk to it.)

    Cheers,
    Marcus

    On 18.03.22 19:26, david vanhorn wrote:
    > I'm using a PCB that I designed with an ARM chip, codec, and SD card for 
logging,
    as my
    > data capture platform.
    > Feeding that is a QSD (Tayloe) front end that I designed, specifically 
for the
    630m ham
    > band, converting down to 1kHz differential I and Q signals to the codec, 
which has
    a 105dB
    > SNR.
    > The front end appears to have a 90dB linear dynamic range so far as I can 
measure
    with my
    > equipment. I'll improve that if I can.
    > Once I capture to SD, then I can pull the SD and process on the PC to 
develop weak
    signal
    > detection.
    >
    >
    >
    > On Fri, Mar 18, 2022 at 12:12 PM Marcus Müller <mmueller@gnuradio.org
    > <mailto:mmueller@gnuradio.org>> wrote:
    >
    >     Hey :)
    >
    >     CSV might or might not be convenient, but if C or assembler is your 
tool: The
    things that
    >     the GNU Radio file source reads or the file sink writes is exactly 
what you
    get when you
    >     take a buffer of samples and do an `fwrite` on that :) Just a dump of 
the raw
    memory to a
    >     file. 32 bit unsigned should be directly digestible by GNU Radio 
(even if
    there were
    >     endianness issues – you can just read as bytes and reshuffle as 
needed :)).
    >
    >     I didn't fully get how you're currently interfacing your hardware. 
Care to
    explain in a
    >     bit more breadth? What are the components of your system, and how 
does the
    computer
    >     running GNU Radio relate?
    >
    >     Best and slightly excited regards,
    >     Marcus
    >
    >     On 18.03.22 18:37, david vanhorn wrote:
    >      > Hi!
    >      >
    >      > I'm trying to interface some radio hardware I built to GnuRadio by 
way of data
    >     captured to
    >      > SD cards.
    >      > I have two channels (I and Q) of 32 bit unsigned data internally, 
and I
    originally
    >     assumed
    >      > CSV would be the easy path, but now I see it's not.
    >      > Coming in through the PC sound card is not an option for me, I'm 
using a
    particular
    >     codec
    >      > selected for the application, and my goal is to develop signal 
processing
    >     algorithms to
    >      > then be implemented back on my processor in C or ASM.
    >      >
    >      > I suppose it would be easiest if I rework my hardware to log data 
as if it
    were the
    >      > "Signal Source" block with complex output.
    >      > Where can I see what that looks like at the level of raw data?
    >      >
    >      >
    >      >
    >      >
    >      > On Fri, Mar 18, 2022 at 4:59 AM Marcus Müller 
<mmueller@gnuradio.org
    >     <mailto:mmueller@gnuradio.org>
    >      > <mailto:mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>>> 
wrote:
    >      >
    >      >     Hi David,
    >      >
    >      >     you could write a quick python block that just reads values 
from the
    CSV file
    >     and outputs
    >      >     them. That'd be a very nice, basic exercise, and I think our 
freshly
    overhauled
    >      >     tutorials[1] should bring you there very quickly!
    >      >     If you want help with that, hit us up in this mailing list 
(ideally after
    >     reading the
    >      >     tutorials up to the point of roughly understanding how to write
    (embedded) Python
    >      >     blocks),
    >      >     and tell us more about the data in your CSV files.
    >      >
    >      >     Alternatively, you could also write a converter of CSV to a 
format that GNU
    >     Radio by
    >      >     itself already has a reader for – and the main candidate here 
would
    probably
    >     just be
    >      >     plain
    >      >     raw data files (as e.g. numpy's `ndarray.tofile("filename")` 
does) –
    the File
    >     Source
    >      >     could
    >      >     directly read that. But with our freshly rewrite Wavfile sink 
and source
    >     blocks, we can
    >      >     write and read most audio files, just as well.
    >      >
    >      >     Then your flow graph could do the signal processing you want – 
e.g
    frequency
    >     translation,
    >      >     low-pass filtering… and finally output it to any device that 
you have a
    GNU Radio
    >      >     interface to (e.g., your sound card). The hardware runs at a 
sample
    rate – GNU
    >     Radio
    >      >     itself just tries to feed it as fast as possible. So, the 
signal
    processing in
    >     GNU Radio
    >      >     itself isn't concerned with rate at all!
    >      >
    >      >     Hope this helps,
    >      >     Marcus
    >      >
    >      >     [1] https://tutorials.gnuradio.org 
<https://tutorials.gnuradio.org>
    >     <https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>>
    >      >
    >      >     PS: you'll often find me online, recommending not to use CSV 
as a sample
    >     storage format.
    >      >     I'll do the same to you here, but not because I think it's in 
any way
    invalid
    >     to have
    >      >     data
    >      >     in CSV files; I just want to point out it might be worth 
thinking about
    using
    >     something
    >      >     else. So take this with a "I think it's pretty cool you're doing 
this!".
    >      >
    >      >     That has the reasons that
    >      >     a) unless you're more restricted than "CSV" says, you don't 
know how
    many bits
    >     are there
    >      >     per sample, as numbers might be represented in different 
lengths, so
    seeking
    >     exactly only
    >      >     works by reading and understanding the whole file up to the 
point you
    seek to,
    >      >     b) conversion of floating point numbers to human-readable form 
incurs
    rounding
    >     errors,
    >      >     and
    >      >     that can really wreck your day if you need to rerun *exactly* 
the same
    >     experiment twice,
    >      >     c) printing numbers as text is really inefficient, both 
storage-wise as
    well as
    >     compute
    >      >     wise (which will only matter at higher sampling rates) and 
sometimes,
    but only
    >     sometimes,
    >      >     ( d) people say that CSV is good because it's human-readable, 
but I
    challenge
    >     anyone to
    >      >     read a text file with only 10000 values and be happier about 
that than
    if he
    >     used a tool
    >      >     that displayed the values graphically, zoomably, and then 
allows for
    inspection
    >     of single
    >      >     values once zoomed sufficiently in.)
    >      >
    >      >
    >      >     On 18.03.22 04:55, david vanhorn wrote:
    >      >      > I've done a little with Gnuradio a couple years ago, but 
I'd now like to
    >     apply it to a
    >      >      > serious problem.
    >      >      >
    >      >      > I have a design I'm working on that will output raw data 
that could be
    >     interpreted
    >      >     as an
    >      >      > audio stream centered on 1kHz.  I'd like to work on 
extracting CW
    signals
    >     that are
    >      >     rather
    >      >      > slow, from a rather narrow bandwidth, and see how far down 
into the
    noise I can
    >      >     actually
    >      >      > extract the signals.
    >      >      >
    >      >      > Is there a block that can bring in CSV data from a file at a
    specific rate, and
    >      >     serve as
    >      >      > the input to my CW detection system?
    >      >      >
    >      >      >
    >      >      > --
    >      >      > K1FZY (WA4TPW) SK  9/29/37-4/13/15
    >      >
    >      >
    >      >
    >      > --
    >      > K1FZY (WA4TPW) SK  9/29/37-4/13/15
    >
    >
    >
    > --
    > K1FZY (WA4TPW) SK  9/29/37-4/13/15



--
K1FZY (WA4TPW) SK  9/29/37-4/13/15




reply via email to

[Prev in Thread] Current Thread [Next in Thread]