discuss-gnuradio
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CSV file as input


From: Marcus Müller
Subject: Re: CSV file as input
Date: Fri, 18 Mar 2022 22:28:25 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.6.2

Gotta say, yours sounds like a cool project. In case you (or really, anyone) want to write up something fun for the blog on gnuradio.org, do let us know.

Since this is for a ham band: we did use to have monthly ham meeting group[1], where aside from a strong community/social aspect, we also had technical project show & tells and tutorials. The meetings lately stopped, because nobody to organize topics stepped up, but I think the interest is still there, I guess.

[1] https://wiki.gnuradio.org/index.php/Talk:HamRadio

On 18.03.22 21:44, david vanhorn wrote:
As I develop my software, I'll implement in the ARM, and it will be able to 
work live.

On Fri, Mar 18, 2022 at 2:25 PM Marcus Müller <mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>> wrote:

    Like the reproducibility aspect of going for storage, but it means no live 
signal
    observation :)

    Just for future hardware ideas: with these bitrates, you should be well in 
range of what
    the cheaper TOLSLINK optical transmitter modules [1] and receivers [2] 
could do.

    [1]
    
https://www.digikey.com/en/products/detail/everlight-electronics-co-ltd/PLT237-T10WH/14641724
    
<https://www.digikey.com/en/products/detail/everlight-electronics-co-ltd/PLT237-T10WH/14641724>

    ,
    
https://www.tme.eu/en/details/fcr6842031t/optical-connectors/cliff/otj-1-fcr6842031t/
    
<https://www.tme.eu/en/details/fcr6842031t/optical-connectors/cliff/otj-1-fcr6842031t/>
    [2]
    
https://www.tme.eu/en/details/fcr6842032r/optical-connectors/cliff/orj-3-fcr6842032r/
    
<https://www.tme.eu/en/details/fcr6842032r/optical-connectors/cliff/orj-3-fcr6842032r/>

    On 18.03.22 19:53, Marcus D. Leech wrote:
     > On 2022-03-18 14:48, david vanhorn wrote:
     >> Noise is always an issue.  I could do a serial port over USB, or TTL 
USART, but I
     >> thought that the SD card would be the most quiet, not requiring any 
electrical
     >> connection to the PC.
     >> It also means that I automatically have my recordings available for 
regression
    testing.
     >>
     > Yeah, I thought that your architecture was probably driven by noise 
concerns--630M
    would
     > not be a "forgiving" band in this regard.  I will point out, just as an 
FYI,
     >    that USB-over-fiber extenders do exist, but because they're rather 
"niche",
    they're not
     > cheap at all....
     >
     >
     >>
     >> On Fri, Mar 18, 2022 at 12:32 PM Marcus Müller <mmueller@gnuradio.org
    <mailto:mmueller@gnuradio.org>> wrote:
     >>
     >>     Ah cool! Thanks for clarifying :) This sounds to be a rather nice 
setup,
    analog-wise!
     >>
     >>     Yeah, then just dumping the raw 32bit unsigned to SD Card is 
probably easiest.
     >>
     >>     (by the way, this is << 1Mb/s, so just dumping the raw data over a 
UART or SPI
     >>     interface
     >>     to some serial-to-USB converter might work as well to get the data 
into your
    PC. If
     >>     your
     >>     ARM does have USB2 built-in, then that would also be a rather cool 
thing, but
     >>     knowing the
     >>     varying quality of chip vendor USB hardware abstractions, that 
might or might
    not be
     >>     easy
     >>     to implement :) In both cases, UART/SPI serial output converted to 
USB, or
    native USB,
     >>     you'd probably have to afterwards write a schmall C/C++ driver, so 
that
    SoapySDR or GNU
     >>     Radio directly can talk to it.)
     >>
     >>     Cheers,
     >>     Marcus
     >>
     >>     On 18.03.22 19:26, david vanhorn wrote:
     >>     > I'm using a PCB that I designed with an ARM chip, codec, and SD 
card for
    logging,
     >>     as my
     >>     > data capture platform.
     >>     > Feeding that is a QSD (Tayloe) front end that I designed, 
specifically for the
     >>     630m ham
     >>     > band, converting down to 1kHz differential I and Q signals to the 
codec,
    which has
     >>     a 105dB
     >>     > SNR.
     >>     > The front end appears to have a 90dB linear dynamic range so far 
as I can
    measure
     >>     with my
     >>     > equipment. I'll improve that if I can.
     >>     > Once I capture to SD, then I can pull the SD and process on the 
PC to
    develop weak
     >>     signal
     >>     > detection.
     >>     >
     >>     >
     >>     >
     >>     > On Fri, Mar 18, 2022 at 12:12 PM Marcus Müller 
<mmueller@gnuradio.org
    <mailto:mmueller@gnuradio.org>
     >>     > <mailto:mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>>> 
wrote:
     >>     >
     >>     >     Hey :)
     >>     >
     >>     >     CSV might or might not be convenient, but if C or assembler 
is your
    tool: The
     >>     things that
     >>     >     the GNU Radio file source reads or the file sink writes is 
exactly what you
     >>     get when you
     >>     >     take a buffer of samples and do an `fwrite` on that :) Just a 
dump of
    the raw
     >>     memory to a
     >>     >     file. 32 bit unsigned should be directly digestible by GNU 
Radio (even if
     >>     there were
     >>     >     endianness issues – you can just read as bytes and reshuffle 
as needed :)).
     >>     >
     >>     >     I didn't fully get how you're currently interfacing your 
hardware. Care to
     >>     explain in a
     >>     >     bit more breadth? What are the components of your system, and 
how does the
     >>     computer
     >>     >     running GNU Radio relate?
     >>     >
     >>     >     Best and slightly excited regards,
     >>     >     Marcus
     >>     >
     >>     >     On 18.03.22 18:37, david vanhorn wrote:
     >>     >      > Hi!
     >>     >      >
     >>     >      > I'm trying to interface some radio hardware I built to 
GnuRadio by
    way of data
     >>     >     captured to
     >>     >      > SD cards.
     >>     >      > I have two channels (I and Q) of 32 bit unsigned data 
internally, and I
     >>     originally
     >>     >     assumed
     >>     >      > CSV would be the easy path, but now I see it's not.
     >>     >      > Coming in through the PC sound card is not an option for 
me, I'm using a
     >>     particular
     >>     >     codec
     >>     >      > selected for the application, and my goal is to develop 
signal processing
     >>     >     algorithms to
     >>     >      > then be implemented back on my processor in C or ASM.
     >>     >      >
     >>     >      > I suppose it would be easiest if I rework my hardware to 
log data as
    if it
     >>     were the
     >>     >      > "Signal Source" block with complex output.
     >>     >      > Where can I see what that looks like at the level of raw 
data?
     >>     >      >
     >>     >      >
     >>     >      >
     >>     >      >
     >>     >      > On Fri, Mar 18, 2022 at 4:59 AM Marcus Müller 
<mmueller@gnuradio.org
    <mailto:mmueller@gnuradio.org>
     >>     >     <mailto:mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>>
     >>     >      > <mailto:mmueller@gnuradio.org 
<mailto:mmueller@gnuradio.org>
    <mailto:mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>>>> wrote:
     >>     >      >
     >>     >      >     Hi David,
     >>     >      >
     >>     >      >     you could write a quick python block that just reads 
values from the
     >>     CSV file
     >>     >     and outputs
     >>     >      >     them. That'd be a very nice, basic exercise, and I 
think our freshly
     >>     overhauled
     >>     >      >     tutorials[1] should bring you there very quickly!
     >>     >      >     If you want help with that, hit us up in this mailing 
list
    (ideally after
     >>     >     reading the
     >>     >      >     tutorials up to the point of roughly understanding how 
to write
     >>     (embedded) Python
     >>     >      >     blocks),
     >>     >      >     and tell us more about the data in your CSV files.
     >>     >      >
     >>     >      >     Alternatively, you could also write a converter of CSV 
to a
    format that GNU
     >>     >     Radio by
     >>     >      >     itself already has a reader for – and the main 
candidate here would
     >>     probably
     >>     >     just be
     >>     >      >     plain
     >>     >      >     raw data files (as e.g. numpy's 
`ndarray.tofile("filename")` does) –
     >>     the File
     >>     >     Source
     >>     >      >     could
     >>     >      >     directly read that. But with our freshly rewrite 
Wavfile sink and
    source
     >>     >     blocks, we can
     >>     >      >     write and read most audio files, just as well.
     >>     >      >
     >>     >      >     Then your flow graph could do the signal processing 
you want – e.g
     >>     frequency
     >>     >     translation,
     >>     >      >     low-pass filtering… and finally output it to any 
device that you
    have a
     >>     GNU Radio
     >>     >      >     interface to (e.g., your sound card). The hardware 
runs at a sample
     >>     rate – GNU
     >>     >     Radio
     >>     >      >     itself just tries to feed it as fast as possible. So, 
the signal
     >>     processing in
     >>     >     GNU Radio
     >>     >      >     itself isn't concerned with rate at all!
     >>     >      >
     >>     >      >     Hope this helps,
     >>     >      >     Marcus
     >>     >      >
     >>     >      >     [1] https://tutorials.gnuradio.org
    <https://tutorials.gnuradio.org> <https://tutorials.gnuradio.org
    <https://tutorials.gnuradio.org>>
     >>     >     <https://tutorials.gnuradio.org 
<https://tutorials.gnuradio.org>
    <https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>>>
     >>     >      >
     >>     >      >     PS: you'll often find me online, recommending not to 
use CSV as a
    sample
     >>     >     storage format.
     >>     >      >     I'll do the same to you here, but not because I think 
it's in any way
     >>     invalid
     >>     >     to have
     >>     >      >     data
     >>     >      >     in CSV files; I just want to point out it might be 
worth thinking
    about
     >>     using
     >>     >     something
     >>     >      >     else. So take this with a "I think it's pretty cool 
you're doing
    this!".
     >>     >      >
     >>     >      >     That has the reasons that
     >>     >      >     a) unless you're more restricted than "CSV" says, you 
don't know how
     >>     many bits
     >>     >     are there
     >>     >      >     per sample, as numbers might be represented in 
different lengths, so
     >>     seeking
     >>     >     exactly only
     >>     >      >     works by reading and understanding the whole file up 
to the point you
     >>     seek to,
     >>     >      >     b) conversion of floating point numbers to 
human-readable form incurs
     >>     rounding
     >>     >     errors,
     >>     >      >     and
     >>     >      >     that can really wreck your day if you need to rerun 
*exactly* the
    same
     >>     >     experiment twice,
     >>     >      >     c) printing numbers as text is really inefficient, both
    storage-wise as
     >>     well as
     >>     >     compute
     >>     >      >     wise (which will only matter at higher sampling rates) 
and sometimes,
     >>     but only
     >>     >     sometimes,
     >>     >      >     ( d) people say that CSV is good because it's 
human-readable, but I
     >>     challenge
     >>     >     anyone to
     >>     >      >     read a text file with only 10000 values and be happier 
about that
    than
     >>     if he
     >>     >     used a tool
     >>     >      >     that displayed the values graphically, zoomably, and 
then allows for
     >>     inspection
     >>     >     of single
     >>     >      >     values once zoomed sufficiently in.)
     >>     >      >
     >>     >      >
     >>     >      >     On 18.03.22 04:55, david vanhorn wrote:
     >>     >      >      > I've done a little with Gnuradio a couple years 
ago, but I'd
    now like to
     >>     >     apply it to a
     >>     >      >      > serious problem.
     >>     >      >      >
     >>     >      >      > I have a design I'm working on that will output raw 
data that
    could be
     >>     >     interpreted
     >>     >      >     as an
     >>     >      >      > audio stream centered on 1kHz.  I'd like to work on 
extracting CW
     >>     signals
     >>     >     that are
     >>     >      >     rather
     >>     >      >      > slow, from a rather narrow bandwidth, and see how 
far down
    into the
     >>     noise I can
     >>     >      >     actually
     >>     >      >      > extract the signals.
     >>     >      >      >
     >>     >      >      > Is there a block that can bring in CSV data from a 
file at a
     >>     specific rate, and
     >>     >      >     serve as
     >>     >      >      > the input to my CW detection system?
     >>     >      >      >
     >>     >      >      >
     >>     >      >      > --
     >>     >      >      > K1FZY (WA4TPW) SK  9/29/37-4/13/15
     >>     >      >
     >>     >      >
     >>     >      >
     >>     >      > --
     >>     >      > K1FZY (WA4TPW) SK  9/29/37-4/13/15
     >>     >
     >>     >
     >>     >
     >>     > --
     >>     > K1FZY (WA4TPW) SK  9/29/37-4/13/15
     >>
     >>
     >>
     >> --
     >> K1FZY (WA4TPW) SK  9/29/37-4/13/15
     >



--
K1FZY (WA4TPW) SK  9/29/37-4/13/15



reply via email to

[Prev in Thread] Current Thread [Next in Thread]