discuss-gnuradio
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Discuss-gnuradio] ATSC Decode Issues


From: GhostOp14
Subject: Re: [Discuss-gnuradio] ATSC Decode Issues
Date: Wed, 26 Apr 2017 08:55:17 -0400

Getting this conversation back into the discuss-gnuradio thread.  Looks like on Kali/debian linux running the latest updates that for some reason the PFB arbitrary resampler is producing the correct number of output samples but incorrect values (confirmed on both Ubuntu and Windows where it works).

Also in testing the atsctest.cfile I found that if you install the gr-correctiq module and run the input through that first, you get much more packet data out the other side (500+MB [I stopped it] versus 80 MB without it).  For anyone else working on ATSC decoding it may help improve the quality of the output IQ stream.

---------- Forwarded message ----------

Sylvain, I've done some more debugging and the mystery only deepens.  I Ran the flowgraph in an Ubuntu 14.04LTS VM with both 3.7.10.1 and a fully pybombs updated version and the output worked fine.  My original test system is Kali / debian linux fully updated.  I added some prints in the pfb_arb_resampler to look at the taps that are used and the numbers all match, however the output bytes are still different.  The debugging is getting a bit deeper than my expertise into the filters and pfb code.  Nothing jumped out at me as to why the output calculations would be different.

Would you be able to fire up a Kali VM to reproduce it?  This will get you up and running pretty quickly... There's 2 quick things to do while installing on Kali.  First, before starting run 'apt-get install zlib1g-dev'.  It's a pre-req that didn't seem to be verified for one of the header files.  Next, when pybombs gets to Apache thrift, when it starts to build break the install and edit <prefix>/src/apache-thrift/lib/cpp/src/thrift/transport/TSSLSocket.cpp.  Search for 'SSLv3_method' and change it to 'SSLv23_method'.  Go into the src directory and manually finish the apache thrift build then go back to the pybombs install and everything will run through fine. 

Now that I know the combo to consistently reproduce the issue that should help hunt it down.

Thanks!
---------- Forwarded message ----------
From: GhostOp14 <address@hidden>
Date: Mon, Apr 24, 2017 at 7:12 AM
Subject: Re: [Discuss-gnuradio] ATSC Decode Issues
To: Ron Economos <address@hidden>


I was playing with the no_pfb version of the flowgraph again now trying to work on getting an input signal to a file and I had a couple of other questions about the ATSC process.  Out of curiosity when you removed the PFB resampler you used an FFT filter rather than a FIR filter for the RRC filter.  From a signal-processing perspective what makes the FFT the correct choice there rather than a FIR?

I was also trying to see if I could adjust the input rate to that 10.7622 Msps with a different resampler if I wanted to use the Airspy instead of my hackrf.  If I drop a rational resampler after the throttle block and use the interpolation/decimation to adjust the output rate to 10.7622 I don't get good output (it was worth a shot right?).  Any math thought behind why that approach doesn't work?

Thanks!


On Sun, Apr 23, 2017 at 4:13 PM, GhostOp14 <address@hidden> wrote:
I sent an email to you and Sylvan to see if he could figure it out.

In the meantime if you want to see where I'm trying to go, if you take the working flowgraph that successfully produces the output file and go up to github and download/install this OOT module I'm working on: https://github.com/ghostop14/gr-atsc2.git

Drop the ATSC2 Streaming Server block in place of the output file, set a port number like 8800 and start the flowgraph.  Then open VLC, go under Media, select "Open Network Stream", put in http://<IP running the flowgraph>:8800/  and voila!  Streaming output.  With the non-PFB flowgraph I would suspect it'll work in real-time.  The TS Stream server block is a really low utilization block.



On Fri, Apr 21, 2017 at 10:58 PM, Ron Economos <address@hidden> wrote:

Bummer! I do have something you can try. It's possible to eliminate the PFB resampler from the flow. I've attached a flow graph that uses just an RRC filter. The input file sample rate has to be 10.76 Msps, but you can create that yourself with the ATSC transmitter (file_atsc_tx.grc).

The test TS file for the ATSC transmitter is here:

http://www.w6rz.net/advatsc.ts

256,952,572 bytes.

BTW, without the PFB resampler the flow runs much faster. Real-time on my modest E5-1607. It's too bad the Airspy can't do fractional sample rates.

That reminds me. On your original flow graph, you had a rational resampler. That's not needed, just set the sample_rate to 10 Msps and let the PFB resampler do all the work. Also, the low-pass filter isn't necessary either.

Ron

On 04/21/2017 09:04 AM, GhostOp14 wrote:
I have another clue I hope will be useful.  So I installed the latest 3.7.11 windows binary on another box and the resulting .ts file actually loaded and played.  Same flowgraph, same input file.  So I looked at the output files with 3.7.12-git/linux and the 3.7.11 and they're the exact same byte size but a diff shows that they differ.  (just another data point.... the linux install is the latest kali: uname -a: "4.9.0-kali3-amd64 #1 SMP Debian 4.9.18-1kali1 (2017-04-04) x86_64 GNU/Linux")

The Windows output file is here:
https://drive.google.com/open?id=0B4VY3GppPdmKcF81UVRrMm9CUmc

The linux output file is here:
https://drive.google.com/open?id=0B4VY3GppPdmKaFpFSDhGTnZuNWM


On Fri, Apr 21, 2017 at 9:08 AM, GhostOp14 <address@hidden> wrote:
No luck on the version swapout.  I uninstalled/purged all the repo gnuradio components and went over to the pybombs version (now reports 3.7.12git-94-gb44e38ce).  No errors on build or startup.  GNURadio and the flowgraph run clean, but the mpeg file still seems corrupt with the same symptoms. 

If it helps, I uploaded the new output file here https://drive.google.com/open?id=0B4VY3GppPdmKYTRzRVdubE5BN1k.  The new one is just slightly bigger which tells me it did something different :)




On Fri, Apr 21, 2017 at 12:13 AM, Ron Economos <address@hidden> wrote:

You nailed it. That arbitrary resampler commit fixes things. You'll have to upgrade to 3.7.11 or 3.7.10.2.

Ron

On 04/20/2017 06:41 PM, GhostOp14 wrote:
Process of elimination?  Same input file, same flowgraph, different output.  I'm running gnuradio 3.7.10.1.  I think I saw a release note on 3.7.10 with the arbitrary resampler putting out too many samples then a bunch of ATSC enhancements it looks like you made for the 3.7.11 release. 

So I guess the obvious question becomes: Could it be the version?

On Thu, Apr 20, 2017 at 9:01 PM, Ron Economos <address@hidden> wrote:

Well, it's definitely not a valid stream. I'm not sure what's going on, since it works fine here.

Ron

On 04/19/2017 05:25 PM, GhostOp14 wrote:

On Wed, Apr 19, 2017 at 7:42 PM, Ron Economos <address@hidden> wrote:

Can you post your TS file output somewhere?

The oversampled_rate is used in the ATSC Receiver FPLL and ATSC Receiver SYNC blocks. I had to change it in the ATSC Receiver SYNC block for things to work.

Ron

On 04/19/2017 04:27 PM, GhostOp14 wrote:
Very interesting.  So I ran the test file through the flowgraph you attached to produce the mpeg.ts file and it did produce a 77M output file, but same thing when I try to open it with VLC or view it with ffmpeg.  When you mentioned the oversampling rates didn't work for you, was that the one in the ATSC RX FIlter?  Did you change that value to something other than 1.1 to get a working ts output file?

On Wed, Apr 19, 2017 at 6:31 PM, Ron Economos <address@hidden> wrote:

I have a test IQ file here:

http://www.w6rz.net/atsctest.cfile

It's 1,656,725,432 bytes.

It's at 6.25 Msps. I've changed your flow graph to work with it (attached).

Aside from changing the sample rate from 6.2 to 6.25 Msps (processing_rate in your flow graph), I put the oversampled_rate back in. The exact value of oversampled_rate makes a difference and the hard coded values you were using don't work.

Ron

On 04/19/2017 02:03 PM, GhostOp14 wrote:
Thanks for responding Ron!  It's actually running on a new i7-6700 CPU @ 3.40GHz.  4 cores, 8 virtual cores.  CPU's only running at about 45%.  I moved it to that from an older i7 laptop because I ran into exactly that case, the CPU couldn't keep up.  Running on the i7-6700 there's no overruns at all in the flowgraph and I do get decoded bytes through the process.   I did also try recording with gqrx then playing the file in.  Same result.  But now that you mention it, I have seen other instances where even though the data's there the timing is a bit off, like a propagation delay.

Do you happen to have a recorded ATSC stream you could pipe through the flowgraph I had attached to see if it works for you from a file?  If it does it may point back to a signal quality issue (back to antennas and amps).

Thanks!

On Wed, Apr 19, 2017 at 4:33 PM, Ron Economos <address@hidden> wrote:

Unless you have a many core CPU (greater than four), the ATSC decoder usually doesn't run in real time. You have to capture an IQ stream to disk first and then demodulate it in non real-time.

Ron

On 04/19/2017 09:14 AM, Ghost Op wrote:
Hi all,

I'm having an issue getting ATSC to decode and I was hoping someone
who has done it successfully may have some pointers.

Here's the scenario.... Basic SDR antenna connected to an LNA4ALL
feeding into an Airspy sampling at 10 MSPS using the attached
flowgraph.  I get a TS file and don't get any atsc_fs_checker error
warnings while capturing, however the TS file doesn't decode with vlc,
ffmpeg, or tsreport.  All messages indicate it can't find the codec in
the stream which would explain why it can't decode the packets.

I've double-checked the local channel frequency (on HDTV ch 34 with a
center freq of 593 MHz).  The raw signal looks good as does the
waterfall for an ATSC signal.  I've tried capturing with and without
the initial low-pass filter in the attached flowgraph, I've tried
using a HackRF at 6.2 MSPS instead of the Airspy at 10 MSPS, tried a
true HDTV antenna (although it's 75 Ohm), and horizontal and vertical
polarizations on the standard SDR antenna to get the cleanest
frequency plot with the best visual SNR.

I get there can be some bad packets in any data stream, but for the
life of me I can't figure out if there's no errors coming from the
decoder and it's decoding successfullly (i.e. producing data with no
warnings) why the stream isn't readable.

Anyone have any suggestions?

Thanks!


_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
_______________________________________________ Discuss-gnuradio mailing list address@hidden https://lists.gnu.org/mailman/listinfo/discuss-gnuradio




reply via email to

[Prev in Thread] Current Thread [Next in Thread]