|
| From: | david vanhorn |
| Subject: | Re: CSV file as input |
| Date: | Fri, 18 Mar 2022 14:44:12 -0600 |
Like the reproducibility aspect of going for storage, but it means no live signal
observation :)
Just for future hardware ideas: with these bitrates, you should be well in range of what
the cheaper TOLSLINK optical transmitter modules [1] and receivers [2] could do.
[1]
https://www.digikey.com/en/products/detail/everlight-electronics-co-ltd/PLT237-T10WH/14641724
, https://www.tme.eu/en/details/fcr6842031t/optical-connectors/cliff/otj-1-fcr6842031t/
[2] https://www.tme.eu/en/details/fcr6842032r/optical-connectors/cliff/orj-3-fcr6842032r/
On 18.03.22 19:53, Marcus D. Leech wrote:
> On 2022-03-18 14:48, david vanhorn wrote:
>> Noise is always an issue. I could do a serial port over USB, or TTL USART, but I
>> thought that the SD card would be the most quiet, not requiring any electrical
>> connection to the PC.
>> It also means that I automatically have my recordings available for regression testing.
>>
> Yeah, I thought that your architecture was probably driven by noise concerns--630M would
> not be a "forgiving" band in this regard. I will point out, just as an FYI,
> that USB-over-fiber extenders do exist, but because they're rather "niche", they're not
> cheap at all....
>
>
>>
>> On Fri, Mar 18, 2022 at 12:32 PM Marcus Müller <mmueller@gnuradio.org> wrote:
>>
>> Ah cool! Thanks for clarifying :) This sounds to be a rather nice setup, analog-wise!
>>
>> Yeah, then just dumping the raw 32bit unsigned to SD Card is probably easiest.
>>
>> (by the way, this is << 1Mb/s, so just dumping the raw data over a UART or SPI
>> interface
>> to some serial-to-USB converter might work as well to get the data into your PC. If
>> your
>> ARM does have USB2 built-in, then that would also be a rather cool thing, but
>> knowing the
>> varying quality of chip vendor USB hardware abstractions, that might or might not be
>> easy
>> to implement :) In both cases, UART/SPI serial output converted to USB, or native USB,
>> you'd probably have to afterwards write a schmall C/C++ driver, so that SoapySDR or GNU
>> Radio directly can talk to it.)
>>
>> Cheers,
>> Marcus
>>
>> On 18.03.22 19:26, david vanhorn wrote:
>> > I'm using a PCB that I designed with an ARM chip, codec, and SD card for logging,
>> as my
>> > data capture platform.
>> > Feeding that is a QSD (Tayloe) front end that I designed, specifically for the
>> 630m ham
>> > band, converting down to 1kHz differential I and Q signals to the codec, which has
>> a 105dB
>> > SNR.
>> > The front end appears to have a 90dB linear dynamic range so far as I can measure
>> with my
>> > equipment. I'll improve that if I can.
>> > Once I capture to SD, then I can pull the SD and process on the PC to develop weak
>> signal
>> > detection.
>> >
>> >
>> >
>> > On Fri, Mar 18, 2022 at 12:12 PM Marcus Müller <mmueller@gnuradio.org
>> > <mailto:mmueller@gnuradio.org>> wrote:
>> >
>> > Hey :)
>> >
>> > CSV might or might not be convenient, but if C or assembler is your tool: The
>> things that
>> > the GNU Radio file source reads or the file sink writes is exactly what you
>> get when you
>> > take a buffer of samples and do an `fwrite` on that :) Just a dump of the raw
>> memory to a
>> > file. 32 bit unsigned should be directly digestible by GNU Radio (even if
>> there were
>> > endianness issues – you can just read as bytes and reshuffle as needed :)).
>> >
>> > I didn't fully get how you're currently interfacing your hardware. Care to
>> explain in a
>> > bit more breadth? What are the components of your system, and how does the
>> computer
>> > running GNU Radio relate?
>> >
>> > Best and slightly excited regards,
>> > Marcus
>> >
>> > On 18.03.22 18:37, david vanhorn wrote:
>> > > Hi!
>> > >
>> > > I'm trying to interface some radio hardware I built to GnuRadio by way of data
>> > captured to
>> > > SD cards.
>> > > I have two channels (I and Q) of 32 bit unsigned data internally, and I
>> originally
>> > assumed
>> > > CSV would be the easy path, but now I see it's not.
>> > > Coming in through the PC sound card is not an option for me, I'm using a
>> particular
>> > codec
>> > > selected for the application, and my goal is to develop signal processing
>> > algorithms to
>> > > then be implemented back on my processor in C or ASM.
>> > >
>> > > I suppose it would be easiest if I rework my hardware to log data as if it
>> were the
>> > > "Signal Source" block with complex output.
>> > > Where can I see what that looks like at the level of raw data?
>> > >
>> > >
>> > >
>> > >
>> > > On Fri, Mar 18, 2022 at 4:59 AM Marcus Müller <mmueller@gnuradio.org
>> > <mailto:mmueller@gnuradio.org>
>> > > <mailto:mmueller@gnuradio.org <mailto:mmueller@gnuradio.org>>> wrote:
>> > >
>> > > Hi David,
>> > >
>> > > you could write a quick python block that just reads values from the
>> CSV file
>> > and outputs
>> > > them. That'd be a very nice, basic exercise, and I think our freshly
>> overhauled
>> > > tutorials[1] should bring you there very quickly!
>> > > If you want help with that, hit us up in this mailing list (ideally after
>> > reading the
>> > > tutorials up to the point of roughly understanding how to write
>> (embedded) Python
>> > > blocks),
>> > > and tell us more about the data in your CSV files.
>> > >
>> > > Alternatively, you could also write a converter of CSV to a format that GNU
>> > Radio by
>> > > itself already has a reader for – and the main candidate here would
>> probably
>> > just be
>> > > plain
>> > > raw data files (as e.g. numpy's `ndarray.tofile("filename")` does) –
>> the File
>> > Source
>> > > could
>> > > directly read that. But with our freshly rewrite Wavfile sink and source
>> > blocks, we can
>> > > write and read most audio files, just as well.
>> > >
>> > > Then your flow graph could do the signal processing you want – e.g
>> frequency
>> > translation,
>> > > low-pass filtering… and finally output it to any device that you have a
>> GNU Radio
>> > > interface to (e.g., your sound card). The hardware runs at a sample
>> rate – GNU
>> > Radio
>> > > itself just tries to feed it as fast as possible. So, the signal
>> processing in
>> > GNU Radio
>> > > itself isn't concerned with rate at all!
>> > >
>> > > Hope this helps,
>> > > Marcus
>> > >
>> > > [1] https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>
>> > <https://tutorials.gnuradio.org <https://tutorials.gnuradio.org>>
>> > >
>> > > PS: you'll often find me online, recommending not to use CSV as a sample
>> > storage format.
>> > > I'll do the same to you here, but not because I think it's in any way
>> invalid
>> > to have
>> > > data
>> > > in CSV files; I just want to point out it might be worth thinking about
>> using
>> > something
>> > > else. So take this with a "I think it's pretty cool you're doing this!".
>> > >
>> > > That has the reasons that
>> > > a) unless you're more restricted than "CSV" says, you don't know how
>> many bits
>> > are there
>> > > per sample, as numbers might be represented in different lengths, so
>> seeking
>> > exactly only
>> > > works by reading and understanding the whole file up to the point you
>> seek to,
>> > > b) conversion of floating point numbers to human-readable form incurs
>> rounding
>> > errors,
>> > > and
>> > > that can really wreck your day if you need to rerun *exactly* the same
>> > experiment twice,
>> > > c) printing numbers as text is really inefficient, both storage-wise as
>> well as
>> > compute
>> > > wise (which will only matter at higher sampling rates) and sometimes,
>> but only
>> > sometimes,
>> > > ( d) people say that CSV is good because it's human-readable, but I
>> challenge
>> > anyone to
>> > > read a text file with only 10000 values and be happier about that than
>> if he
>> > used a tool
>> > > that displayed the values graphically, zoomably, and then allows for
>> inspection
>> > of single
>> > > values once zoomed sufficiently in.)
>> > >
>> > >
>> > > On 18.03.22 04:55, david vanhorn wrote:
>> > > > I've done a little with Gnuradio a couple years ago, but I'd now like to
>> > apply it to a
>> > > > serious problem.
>> > > >
>> > > > I have a design I'm working on that will output raw data that could be
>> > interpreted
>> > > as an
>> > > > audio stream centered on 1kHz. I'd like to work on extracting CW
>> signals
>> > that are
>> > > rather
>> > > > slow, from a rather narrow bandwidth, and see how far down into the
>> noise I can
>> > > actually
>> > > > extract the signals.
>> > > >
>> > > > Is there a block that can bring in CSV data from a file at a
>> specific rate, and
>> > > serve as
>> > > > the input to my CW detection system?
>> > > >
>> > > >
>> > > > --
>> > > > K1FZY (WA4TPW) SK 9/29/37-4/13/15
>> > >
>> > >
>> > >
>> > > --
>> > > K1FZY (WA4TPW) SK 9/29/37-4/13/15
>> >
>> >
>> >
>> > --
>> > K1FZY (WA4TPW) SK 9/29/37-4/13/15
>>
>>
>>
>> --
>> K1FZY (WA4TPW) SK 9/29/37-4/13/15
>
| [Prev in Thread] | Current Thread | [Next in Thread] |