Re: [Discuss-gnuradio] Randomness Introduced in Gnuradio WAV Source Block?

[Tommy Tracy II]

On Tue, Jan 15, 2013 at 01:48:47AM -0500, Tommy Tracy II wrote:

I eventually determined, that the randomness was added somewhere in the WAV
Source block. Has anyone seen this?

Tommy,

Martin,

can you please post more details:
- Which GR version?

$gnuradio-config-info -v
Ubuntu: v3.6.2-265-gb15b38df

Gentoo: v3.6.x-xxx-xunknown

The version of gnu radio installed on the Gentoo box was installed from source from the latest stable release of gnuradio-3.6.3.

As per some prior posts, I get failed 'make test' results, but the install completes.

- What format are your WAV files? (8/16 Bit, Mono/Stereo)

This is an 8 Bit/Sample, 11025Hz, Mono WAV file.

 - Does that matter?

I don't think it should. There is an 8-bit to float (32-bit) conversion somewhere in that block, but it should be deterministic. It appears to be deterministic on the Ubuntu machine.

- How large is the WAV file?

The WAV file is 10.9 MBytes

- To be clear, this works on Ubuntu, but not on Gentoo? (works on my
 Ubuntu …)

Yes. This is how I determined it 'worked'.

for i in 1 2 3 4 5; do ./WAV_TEST.py ../sources/<source.wav> $i; done

for i in 1 2 3 4 5; do md5sum $i; done

Ubuntu: They are the same

143a5b3be71e8476ff2f5edd1086e79f 1 

143a5b3be71e8476ff2f5edd1086e79f 2 

143a5b3be71e8476ff2f5edd1086e79f 3 

143a5b3be71e8476ff2f5edd1086e79f 4 

143a5b3be71e8476ff2f5edd1086e79f 5

Gentoo: They are not

6f328ef003b3a60cf4b9675a5453d142 1 

c544a257aef0a2f95c9cda82f92acbd6 2

c544a257aef0a2f95c9cda82f92acbd6 3 

6f328ef003b3a60cf4b9675a5453d142 4 

cf9dc9f7d0c80a3c205c47c60c6c7ac9 5

Interesting note though; all 10 files have the same size.

- Do you know how the output is corrupted?

This is where it gets weird.

for i in 1 2 3 4 5; do hexdump $i > $i.hex; done

To compare, I looked at a few lines of 1.hex, 2.hex, and 5.hex (they are all different)

1.hex

0000000 a27e a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

0000010 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

*

0000170 a1a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

0000180 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

*

…

0019710 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a3

0019720 a1a1 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

0019730 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

*

001a1d0 a2a3 a1a1 a3a3 a1a2 a2a2 a2a2 a2a2 a2a2

001a1e0 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

…

04f09a0 a1a1 a3a3 a1a2 a3a2 a1a2 a2a1 a2a3 a2a1

04f09b0 a3a3 a1a1 a3a2 a1a2 a3a1 a2a3 a2a1 a2a3

04f09c0 a1a1 a3a2 a1a2 a3a2 a1a3 a2a1 a2a3 a1a1

04f09d0 a3a3 a1a2 a3a2 a2a2 a2a1 a2a3 a2a1 a3a3

2.hex

0000000 421e 4242 4242 4242 4242 4242 4242 4242

0000010 4242 4242 4242 4242 4242 4242 4242 4242

*

0000170 4142 4242 4242 4242 4242 4242 4242 4242

0000180 4242 4242 4242 4242 4242 4242 4242 4242

*

…

0019710 4242 4242 4242 4242 4242 4242 4242 4243

0019720 4141 4242 4242 4242 4242 4242 4242 4242

0019730 4242 4242 4242 4242 4242 4242 4242 4242

*

001a1d0 4243 4141 4343 4142 4242 4242 4242 4242

001a1e0 4242 4242 4242 4242 4242 4242 4242 4242

*

…

04f09a0 4141 4343 4142 4342 4142 4241 4243 4241

04f09b0 4343 4141 4342 4142 4341 4243 4241 4243

04f09c0 4141 4342 4142 4342 4143 4241 4243 4141

5.hex

0000000 c29e c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2

0000010 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2

*

0000170 c1c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2

0000180 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2

*

…

0019710 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c3

0019720 c1c1 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2

0019730 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2

*

001a1d0 c2c3 c1c1 c3c3 c1c2 c2c2 c2c2 c2c2 c2c2

001a1e0 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2 c2c2

*

…

04f09a0 c1c1 c3c3 c1c2 c3c2 c1c2 c2c1 c2c3 c2c1

04f09b0 c3c3 c1c1 c3c2 c1c2 c3c1 c2c3 c2c1 c2c3

04f09c0 c1c1 c3c2 c1c2 c3c2 c1c3 c2c1 c2c3 c1c1

Other than the first 2-byte entry (which seems to take the form of <random>2<random>e, all other bytes seem to be identical with a simple substitution.

This little program shows where the problem may be. The input is a WAV file.
The float stream is converted to chars, and then dumped to a file. This should
be completely reproducible, but it doesn't seem to be in the case of my Gentoo
machine.

Are the Ubuntu & Gentoo machine different hardware?

Yes. They are very different.

The Ubuntu machine is a core i5, 2GB desktop.

The gentoo machine is a storage-less single-board computer, with NFS filesystem.

[code]

if __name__ == '__main__':
       tb = top_block()
       tb.run(True)

Hm, I'm not even sure what the 'True' does here.. normally you limit the
FG to a number of items with the run() argument.

I'm not sure either… The 'True' does slow it down considerably. I removed it and still get the same behavior.

Also, can you please run this test: Add a throttle after the file
source, and set it to a low value (50k or something). Then run again.
(I'm serious :).

Updated Throttle Code:

---

class top_block(gr.top_block):

        def __init__(self):

                gr.top_block.__init__(self, "WAV TEST")

                ##################################################

                # Blocks

                ##################################################

                inputFileName = str(sys.argv[1])

                outputFileName = str(sys.argv[2])

                self.gr_wavfile_source_0 = gr.wavfile_source(inputFileName, False)

                self.throttle = gr.throttle(gr.sizeof_float, 50000)

                self.gr_float_to_uchar_0 = gr.float_to_uchar()

                self.gr_file_sink_0 = gr.file_sink(gr.sizeof_char*1, outputFileName)

                #self.gr_file_sink_0.set_unbuffered(False)

                ##################################################

                # Connections

                ##################################################

                #self.connect((self.gr_wavfile_source_0,0), (self.gr_float_to_uchar_0, 0))      

                self.connect((self.gr_wavfile_source_0, 0), (self.throttle, 0))

                self.connect((self.throttle, 0), (self.gr_float_to_uchar_0, 0))

                self.connect((self.gr_float_to_uchar_0, 0), (self.gr_file_sink_0, 0))

if __name__ == '__main__':

        tb = top_block()

        start = time()

        tb.run()

        end = time()

        print str(end-start)

---

As expected, the throttle slowed down the execution considerably.

But… after hex dumping the new files, I get the following:

(1-5 are done without the throttle; 6-10 are done with the throttle)

$ ls -sh *.hex

3.6M 1.hex   3.6M 2.hex  3.6M 4.hex  4.0K 6.hex  4.0K 8.hex

4.0K 10.hex  3.6M 3.hex  3.6M 5.hex  3.6M 7.hex  3.6M 9.hex

You'll notice that some of the resulting hex files for 6-10 are much smaller…

Ignoring the hex-dump failures…

7.hex

0000000 bebe bebe bebe bebe bebe bebe bebe bebe

*

0000170 bdbe bebe bebe bebe bebe bebe bebe bebe

0000180 bebe bebe bebe bebe bebe bebe bebe bebe

*

…

0019710 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e3

0019720 e1e1 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2

0019730 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2

*

001a1d0 e2e3 e1e1 e3e3 e1e2 e2e2 e2e2 e2e2 e2e2

001a1e0 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2 e2e2

*

…

04f09a0 e1e1 e3e3 e1e2 e3e2 e1e2 e2e1 e2e3 e2e1

04f09b0 e3e3 e1e1 e3e2 e1e2 e3e1 e2e3 e2e1 e2e3

04f09c0 e1e1 e3e2 e1e2 e3e2 e1e3 e2e1 e2e3 e1e1

The difference in 7.hex is that lines 0000000-0001c70 are padded with bebe; from there on, I get the same e<1,2,3>e<1,2,3> bytes.

9.hex

0000000 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e

*

0000170 7d7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e

0000180 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e 7e7e

*

...

0019710 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a3

0019720 a1a1 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

0019730 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

*

001a1d0 a2a3 a1a1 a3a3 a1a2 a2a2 a2a2 a2a2 a2a2

001a1e0 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2 a2a2

*

...

04f09a0 a1a1 a3a3 a1a2 a3a2 a1a2 a2a1 a2a3 a2a1

04f09b0 a3a3 a1a1 a3a2 a1a2 a3a1 a2a3 a2a1 a2a3

04f09c0 a1a1 a3a2 a1a2 a3a2 a1a3 a2a1 a2a3 a1a1

The difference in 9.hex is that lines 0000000-0001c70 are padded with 7e7e; from there on, I get the same a<1,2,3>a<1,2,3> bytes as shown in 1.hex

MB

--
Karlsruhe Institute of Technology (KIT)
Communications Engineering Lab (CEL)

Dipl.-Ing. Martin Braun
Research Associate

Kaiserstraße 12
Building 05.01
76131 Karlsruhe

Phone: +49 721 608-43790
Fax: +49 721 608-46071
www.cel.kit.edu

KIT -- University of the State of Baden-Württemberg and
National Laboratory of the Helmholtz Association
_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio

                   Sincerely,

          Tommy James Tracy II

          PhD Student

High Performance Low Power Lab 

           University of Virginia