RE: [Discuss-gnuradio] information theory -- follow up (off topic )

[Brian_Whitaker]

== snippets selected from one of Matt Ettus' emails ====
== /* my comments like this */ ====

Basically, nobody is saying there is anything here which contradicts
Shannon. 
You just need to realize what Shannon's theorem says and what it doesn't. 
Shannon's theorem, for example says nothing about interference -- it talks
about AWGN.  A very strong interferer is no big deal if you can filter it
away, right?
 It says nothing about antennas, shared channels, path loss, spatial
diveristy, directional diversity, etc.

/* (bw) I absolutely agree, except I don't know what you mean by 'shared
channels'... Matt's comment here about interfers supports the idea of there
not being any 'inherent interference' in signals of different frequencies.
If you can't see a weak blue light in the presence of a super-bright violet
one, its because you need a better filter mechanism.  This is referred to as
an out-of-channel jammer, or out-of-band jammer problem.
  The on-channel jammer issue (strong/weak blue light given earlier) is
another issue -- here you need some other mechanism to distinguish the two
signals... if you know the jammer is on all the time, and the desired is
pulsing, then you look for the AM sidelobes... otherwise you hope you can
fall back on spatial/directional diversity or some other way to separate the
two.
  I agree that just because a jammer is on-frequency with our desired signal
does not mean that we can't decode the desired. 
  If I understand correctly, the basis of Dave's thoughts referred to in the
Salon article is that we need modern-day systems to use more sophisticated
tricks to encode channels than just "you get this 30kHz, and you get this
other 30kHz" -- the classic FDMA approach of 100 years ago. 
*/

The idea is that given a network in which everyone talks at will on the same
freqency, the SNR can become significantly negative.  This does not preclude
communication, it merely requires a different way of communicating.  We know
this intuitively if we've ever been to a football game.  Everyone is talking
at the same time, in the audio band.  There is more interference than
signal.  Yet we can still communicate.  (This observation/metaphor is due to
Tim Shepard, BTW)

/* Not a fair representation of what we're talking about -- it is not
straight forward to describe the information transferred when talking over a
crowd in a football game. Furthermore, you have many other cues to help you
decode the words spoken, if we restrict our definition to this -- facial
expressions, known speach patterns from a familiar person. All of this is a
form of spreading -- you're including lots of known/predictable 'stuff'
along with the actual 'message' trying to transmit, and in doing so, you can
listen well under the system noise floor (ie, negative SNR).
  Example:  You hear your wife singing a familiar song -- at one point, you
can't hear a damn thing, but see her lips mouth the word
"supercalifragilisticexpealladocious" so you know that's what she said...
alternatively, you listen to two Japanese fellows with heavy accents (seated
in front of you) discuss their company's business -- at an SNR of 3.0dB you
still might not be able to make out what is being said.
  It all comes down to how 'information' and SNR is defined -- Shannon had a
very particular definition where we have to reduce the symbol set to remove
redundancy and such -- difficult to do in discussions and analogies like
this
*/

We also know this from spread spectrum systems like CDMA cellphones.  The
difference is that we are talking about much bigger networks here, and they
are
decentralized.

There seems to be the belief that SNR would become "too negative to
communicate".  This is false.  It is the same as Olber's paradox -- if there
are
infinitely many stars in an infinite universe, why isn't the sky bright at
night?

So given a non-zero SNR, we can always communicate, albeit slowly, or more
accurately, at fewer bits per second per hertz.  It might be 1 bit per
second
per 100 Hertz.


/* (bw) you mean non-zero SNR as a ratio, not in dB, right?  this is the
part where my head explodes. Shannon tells us that there is indeed minimum
SNR required to support a given bandwidth efficiency (measured in
bits-per-second-per-Hz -- units look like the timed acceleration of a bit,
funny, huh?)... with a theoretical asymptote at -1.6dB -- that is, with less
than this, you can't transmit anything reliably.
  Of course CDMA (and spreading in general) offers you a 'spreading gain' --
for CDMA, its just the ratio of the chip rate to the data rate -- something
like 10dB-20dB. This means that the RAKE receiver can still decode one
channel when it look like its buried 10dB in the noise floor.
  This appears to preclude the idea of not having a fundamental limit on SNR
required until you remember how Shannon talks about 'information' -- these
extra bits we're using to spread the signal, they're NOT information...
they're a known, predictable sequence -- just a mechanism we use to share a
given frequency band to reduce 'deadbands' required by freqency-division
multiple access methods -- in an effort to increase the spectral effiency.
*/


If you are really interested in this, you should check out Tim Shepard's PhD
thesis (its very readable, and not too long).  You can find it at:

ftp://ftp.lcs.mit.edu/pub/lcs-pubs/tr.outbox/MIT-LCS-TR-670.ps.gz

After that, you might want to read my MS thesis which extended Tim's work to
higher-order path-loss environments.  You can find it at
http://ettus.com/thesis.ps.gz

/* Thanks for these links -- I'll be sure to read them. I'm sure these will
provide some insight to help me fill in some of the holes in my
understanding.
*/

Matt

/*
Brian Whitaker
Maxim RF Applications