discuss-gnuradio
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CSV file as input


From: Marcus D. Leech
Subject: Re: CSV file as input
Date: Fri, 18 Mar 2022 16:17:24 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0

On 2022-03-18 15:23, david vanhorn wrote:
Yeah, I have one.  The distal end still makes noise.  Better at galvanic isolation.  It does prevent the PC noise from propagating down the cable.
I did think about using Toslink but that just seemed like another can-o-worms.
Not at all surprising.  Getting digital electronics to be "zero emissions" at low frequencies is really really hard.



On Fri, Mar 18, 2022 at 12:54 PM Marcus D. Leech <patchvonbraun@gmail.com> wrote:
On 2022-03-18 14:48, david vanhorn wrote:
Noise is always an issue.  I could do a serial port over USB, or TTL USART, but I thought that the SD card would be the most quiet, not requiring any electrical connection to the PC.
It also means that I automatically have my recordings available for regression testing.

Yeah, I thought that your architecture was probably driven by noise concerns--630M would not be a "forgiving" band in this regard.  I will point out, just as an FYI,
  that USB-over-fiber extenders do exist, but because they're rather "niche", they're not cheap at all....



On Fri, Mar 18, 2022 at 12:32 PM Marcus Müller <mmueller@gnuradio.org> wrote:
Ah cool! Thanks for clarifying :) This sounds to be a rather nice setup, analog-wise!

Yeah, then just dumping the raw 32bit unsigned to SD Card is probably easiest.

(by the way, this is << 1Mb/s, so just dumping the raw data over a UART or SPI interface
to some serial-to-USB converter might work as well to get the data into your PC. If your
ARM does have USB2 built-in, then that would also be a rather cool thing, but knowing the
varying quality of chip vendor USB hardware abstractions, that might or might not be easy
to implement :) In both cases, UART/SPI serial output converted to USB, or native USB,
you'd probably have to afterwards write a schmall C/C++ driver, so that SoapySDR or GNU
Radio directly can talk to it.)

Cheers,
Marcus

On 18.03.22 19:26, david vanhorn wrote:
> I'm using a PCB that I designed with an ARM chip, codec, and SD card for logging, as my
> data capture platform.
> Feeding that is a QSD (Tayloe) front end that I designed, specifically for the 630m ham
> band, converting down to 1kHz differential I and Q signals to the codec, which has a 105dB
> SNR.
> The front end appears to have a 90dB linear dynamic range so far as I can measure with my
> equipment. I'll improve that if I can.
> Once I capture to SD, then I can pull the SD and process on the PC to develop weak signal
> detection.
>
>
>
> On Fri, Mar 18, 2022 at 12:12 PM Marcus Müller <mmueller@gnuradio.org
> <mailto:mmueller@gnuradio.org>> wrote:
>
>     Hey :)
>
>     CSV might or might not be convenient, but if C or assembler is your tool: The things that
>     the GNU Radio file source reads or the file sink writes is exactly what you get when you
>     take a buffer of samples and do an `fwrite` on that :) Just a dump of the raw memory to a
>     file. 32 bit unsigned should be directly digestible by GNU Radio (even if there were
>     endianness issues – you can just read as bytes and reshuffle as needed :)).
>
>     I didn't fully get how you're currently interfacing your hardware. Care to explain in a
>     bit more breadth? What are the components of your system, and how does the computer
>     running GNU Radio relate?
>
>     Best and slightly excited regards,
>     Marcus
>
>     On 18.03.22 18:37, david vanhorn wrote:
>      > Hi!
>      >
>      > I'm trying to interface some radio hardware I built to GnuRadio by way of data
>     captured to
>      > SD cards.
>      > I have two channels (I and Q) of 32 bit unsigned data internally, and I originally
>     assumed
>      > CSV would be the easy path, but now I see it's not.
>      > Coming in through the PC sound card is not an option for me, I'm using a particular
>     codec
>      > selected for the application, and my goal is to develop signal processing
>     algorithms to
>      > then be implemented back on my processor in C or ASM.
>      >
>      > I suppose it would be easiest if I rework my hardware to log data as if it were the
>      > "Signal Source" block with complex output.
>      > Where can I see what that looks like at the level of raw data?
>      >
>      >
>      >
>      >
>      > On Fri, Mar 18, 2022 at 4:59 AM Marcus Müller <mmueller@gnuradio.org
>     <mailto:mmueller@gnuradio.org>
>      > <mailto:mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>>> wrote:
>      >
>      >     Hi David,
>      >
>      >     you could write a quick python block that just reads values from the CSV file
>     and outputs
>      >     them. That'd be a very nice, basic exercise, and I think our freshly overhauled
>      >     tutorials[1] should bring you there very quickly!
>      >     If you want help with that, hit us up in this mailing list (ideally after
>     reading the
>      >     tutorials up to the point of roughly understanding how to write (embedded) Python
>      >     blocks),
>      >     and tell us more about the data in your CSV files.
>      >
>      >     Alternatively, you could also write a converter of CSV to a format that GNU
>     Radio by
>      >     itself already has a reader for – and the main candidate here would probably
>     just be
>      >     plain
>      >     raw data files (as e.g. numpy's `ndarray.tofile("filename")` does) – the File
>     Source
>      >     could
>      >     directly read that. But with our freshly rewrite Wavfile sink and source
>     blocks, we can
>      >     write and read most audio files, just as well.
>      >
>      >     Then your flow graph could do the signal processing you want – e.g frequency
>     translation,
>      >     low-pass filtering… and finally output it to any device that you have a GNU Radio
>      >     interface to (e.g., your sound card). The hardware runs at a sample rate – GNU
>     Radio
>      >     itself just tries to feed it as fast as possible. So, the signal processing in
>     GNU Radio
>      >     itself isn't concerned with rate at all!
>      >
>      >     Hope this helps,
>      >     Marcus
>      >
>      >     [1] https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>
>     <https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>>
>      >
>      >     PS: you'll often find me online, recommending not to use CSV as a sample
>     storage format.
>      >     I'll do the same to you here, but not because I think it's in any way invalid
>     to have
>      >     data
>      >     in CSV files; I just want to point out it might be worth thinking about using
>     something
>      >     else. So take this with a "I think it's pretty cool you're doing this!".
>      >
>      >     That has the reasons that
>      >     a) unless you're more restricted than "CSV" says, you don't know how many bits
>     are there
>      >     per sample, as numbers might be represented in different lengths, so seeking
>     exactly only
>      >     works by reading and understanding the whole file up to the point you seek to,
>      >     b) conversion of floating point numbers to human-readable form incurs rounding
>     errors,
>      >     and
>      >     that can really wreck your day if you need to rerun *exactly* the same
>     experiment twice,
>      >     c) printing numbers as text is really inefficient, both storage-wise as well as
>     compute
>      >     wise (which will only matter at higher sampling rates) and sometimes, but only
>     sometimes,
>      >     ( d) people say that CSV is good because it's human-readable, but I challenge
>     anyone to
>      >     read a text file with only 10000 values and be happier about that than if he
>     used a tool
>      >     that displayed the values graphically, zoomably, and then allows for inspection
>     of single
>      >     values once zoomed sufficiently in.)
>      >
>      >
>      >     On 18.03.22 04:55, david vanhorn wrote:
>      >      > I've done a little with Gnuradio a couple years ago, but I'd now like to
>     apply it to a
>      >      > serious problem.
>      >      >
>      >      > I have a design I'm working on that will output raw data that could be
>     interpreted
>      >     as an
>      >      > audio stream centered on 1kHz.  I'd like to work on extracting CW signals
>     that are
>      >     rather
>      >      > slow, from a rather narrow bandwidth, and see how far down into the noise I can
>      >     actually
>      >      > extract the signals.
>      >      >
>      >      > Is there a block that can bring in CSV data from a file at a specific rate, and
>      >     serve as
>      >      > the input to my CW detection system?
>      >      >
>      >      >
>      >      > --
>      >      > K1FZY (WA4TPW) SK  9/29/37-4/13/15
>      >
>      >
>      >
>      > --
>      > K1FZY (WA4TPW) SK  9/29/37-4/13/15
>
>
>
> --
> K1FZY (WA4TPW) SK  9/29/37-4/13/15


--
K1FZY (WA4TPW) SK  9/29/37-4/13/15



--
K1FZY (WA4TPW) SK  9/29/37-4/13/15


reply via email to

[Prev in Thread] Current Thread [Next in Thread]