Re: [Discuss-gnuradio] information theory -- follow up (off topic)

[Matt Ettus]

> There is a fundamental assumption in Shannon's model that there is a
> single transmitter and a single receiver and that the noise is
> Gaussian.  All the excitement that David Reed is probably talking
> about (sorry, haven't read the Salon article yet), is related to
> "MIMO" systems, "Multiple Input Multiple Output".

That's not really what MIMO systems are, and I wouldn't say he is talking about
MIMO systems in particular, but the basic idea is right.

MIMO systems are comm systems in which both the transmitter and receiver have
multiple antennas.  The data is encoded in some form, across the antennas which
allows you to take advantage of multipath and other forms of diversity.  This
idea is basically the same as "smart antennas" or BLAST (from bell labs).

What David Reed is talking about is huge networks which share the same spectrum,
and have no global coordination (i.e. they don't take turns talking).  They
might use MIMO (especially if they're smart), but its just another tool in the
box.

Basically, nobody is saying there is anything here which contradicts Shannon. 
You just need to realize what Shannon's theorem says and what it doesn't. 
Shannon's theorem, for example says nothing about interference -- it talks about
AWGN.  A very strong interferer is no big deal if you can filter it away, right?
 It says nothing about antennas, shared channels, path loss, spatial diveristy,
directional diversity, etc.

The idea is that given a network in which everyone talks at will on the same
freqency, the SNR can become significantly negative.  This does not preclude
communication, it merely requires a different way of communicating.  We know
this intuitively if we've ever been to a football game.  Everyone is talking at
the same time, in the audio band.  There is more interference than signal.  Yet
we can still communicate.  (This observation/metaphor is due to Tim Shepard, 
BTW)

We also know this from spread spectrum systems like CDMA cellphones.  The
difference is that we are talking about much bigger networks here, and they are
decentralized.

There seems to be the belief that SNR would become "too negative to
communicate".  This is false.  It is the same as Olber's paradox -- if there are
infinitely many stars in an infinite universe, why isn't the sky bright at 
night?

So given a non-zero SNR, we can always communicate, albeit slowly, or more
accurately, at fewer bits per second per hertz.  It might be 1 bit per second
per 100 Hertz.

Now there are ways to improve that number, which might be necessary for any
practical system, but which are by no means required for the theory to work:

1 -- The interference isn't noise-like, so you could detect it and subtract it
off.  This is a general case of multi-user detection.

2 -- You could use directional antennas.  If you cut off noise from 180 degrees,
you would improve your SNR by a factor of 2, and thus get a factor of 2 in
capacity.

3 -- You could use MIMO systems and take advantage of multipath, or to
adaptively null out interferers.

4 -- You could only transmit over short hops, and use power control.

> The exciting part is that although the capacity between any two nodes
> falls off because of the increase in noise, the aggregate throughput
> of the network continues to grow as you add nodes.  There are usually
> some assumptions about node density and an assumption of 1/(r*r) fall
> off in power.  In reality there are greater losses due to buildings,
> etc, allowing a *higher* aggegrate throughput.  Other things that are
> not intuitively obvious that help the capacity of the system include
> multi-path and motion of the nodes.


Yes.

If you are really interested in this, you should check out Tim Shepard's PhD
thesis (its very readable, and not too long).  You can find it at:

ftp://ftp.lcs.mit.edu/pub/lcs-pubs/tr.outbox/MIT-LCS-TR-670.ps.gz

After that, you might want to read my MS thesis which extended Tim's work to
higher-order path-loss environments.  You can find it at
http://ettus.com/thesis.ps.gz

Matt