Re: [Discuss-gnuradio] Incorrect quantizations when converting from float to char

[Paul Boven]

Hi Marcus,

Just reran the test after editing volk_config, and the result is 
somewhat surprising:

Every float in [-1:1] now converts to zero. Every float in [1:2] now 
converts to 1. Whereas it should be [-0.5:0.5] and [0.5:1.5].

It seems that most of the time, the u_sse2 converter is used, but at the 
end of each multiple of 8192 bytes, a few are done with the 'generic' 
converter - that would match perfectly with the observed behaviour.

It was also pointed out to me (on irc, unfortunately no longer in my 
history) that it is strange that for some acceleration types, there is a 
cast to int16_t instead of int8_t at the end of the routine, e.g.:

https://github.com/gnuradio/volk/blob/master/kernels/volk/volk_32f_s32f_convert_8i.h#L200

I had not really looked into that before because having run the 
volk_profile seemed to make no difference.

Regards, Paul Boven.

On 06/09/2018 06:08 PM, Müller, Marcus (CEL) wrote:
> I can reproduce these, but do the errors disappear for you if you
> replace "u_sse2 u_sse2" with "generic generic" on that line?
>
>
> Best regards,
> Marcus
> On Sat, 2018-06-09 at 18:04 +0200, Paul Boven wrote:
> > Hi Marcus,
> >
> > This machine did not yet have a volk_config when I ran these tests.
> >
> > I have since run volk_profile and rebooted, and the Float->Char
> > quantization bug still occurs.
> >
> > $ volk-config-info --machine
> > avx2_64_mmx_orc
> >
> > $ grep volk_32f_s32f_convert_8i .volk/volk_config
> > volk_32f_s32f_convert_8i u_sse2 u_sse2
> >
> > Regards, Paul Boven.
> >
> > On 06/09/2018 05:30 PM, Müller, Marcus (CEL) wrote:
> > > Hi Paul,
> > >
> > > hm, OK, considering the actual conversion is done in VOLK, can you
> > > tell
> > > us
> > >
> > > * whether ~/.volk/volk_config exists (and if so, its contents
> > > regarding
> > > volk_32f_s32f_convert_8i )
> > > * what the output of `volk-config-info --machine` is?
> > >
> > > Thanks,
> > > Marcus
> > >
> > > On Sat, 2018-06-09 at 17:13 +0200, Paul Boven wrote:
> > > > Hi everyone,
> > > >
> > > > I'm trying to perform 2 bit sampling of an RF signal. In one
> > > > approach,
> > > > I'm using a float->char block, and noticed that around zero, a
> > > > number
> > > > of
> > > > float inputs become quantized in a bin that's one off from the
> > > > correct
> > > > value. The ones that are wrong are always off by one, with their
> > > > quantization error always in the direction of zero.
> > > >
> > > > The problem can be demonstrated with a really simple flowchart,
> > > > using
> > > > the following blocks:
> > > >
> > > > * Noise Source (Noise Type: Gaussian, Amplitude: 1, Seed: 0,
> > > > Output
> > > > type: Float)
> > > > * Throttle
> > > > The throttle is then connected to two blocks:
> > > > * A file-sink (Type Float) and a
> > > > * 'Float to Char' block.
> > > > * The float to char block is again connected to a File Sink, now
> > > > of
> > > > type
> > > > Char.
> > > >
> > > > As an example, I've plotted all the samples that quantized as
> > > > zero.
> > > > These should fall in the range [-0.5:0.5], but occasionally we
> > > > also
> > > > get
> > > > hits that lie within [-1:1]. These mishaps are rare (about one in
> > > > 2000).
> > > > It also shows that they only occur at multiples of 8192 samples,
> > > > and
> > > > zooming in reveals that they always happen shortly before the
> > > > next
> > > > multiple of 8192, not after.
> > > >
> > > > For other values than 0, the same applies, but the
> > > > misquantizations
> > > > are
> > > > only in one direction, ending up in a lower bin if the input is
> > > > positive, or in a higher bin if the input is negative. Again, the
> > > > misquantizations only occur in half the bin: For a value of 1,
> > > > the
> > > > float
> > > > value should be in [0.5:1.5], but occasionally a value in [1.5:2]
> > > > also
> > > > ends up being quantized as 1.
> > > >
> > > > This seems to me a bug that is somehow related to internal block
> > > > boundaries, but I'm not familiar enough with the internals of
> > > > GnuRadio
> > > > to figure out just what's going wrong.
> > > >
> > > > The problem does NOT occur when converting to Short or Int.
> > > >
> > > > This is using GnuRadio 3.7.11 (as packaged with Ubuntu 18.04).
> > > >
> > > > Regards, Paul Boven.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Discuss-gnuradio mailing list
> > > > address@hidden
> > > > https://lists.gnu.org/mailman/listinfo/discuss-gnuradio