Re: [Discuss-gnuradio] MIMO Trigger (was Re: USRP2 questions)

[Matt Ettus]

Robey, Frank wrote:
> Matt,
>
> I would like to suggest that what you have described is necessary but not 
> sufficient for some (most?) MIMO or multi-channel applications and would like 
> to describe the other pieces needed for a more complete solution.
>
> For many applications differential phase shift and time delay between channels 
> that is not observable without known external signals for calibration and that 
> changes every time you reset or power cycle is unacceptable.  To ensure the 
> phase shift and time delay are stable and repeatable the following additional 
> steps are necessary.  These steps are ordered in increasing channel-to-channel 
> precision.
>   

I believe we have done all of these.

In the definitions that I am going by, MIMO and phased-array have 
different requirements. 

   MIMO requires:
       Samples are taken synchronously
       Which samples from each stream were taken at the same time is known
       All frequency conversion operations (i.e. mixing) use local 
oscillators with an arbitrary, possibly unknown, but constant phase 
relationship.

   Phased arrays require everything that MIMO requires PLUS:
      The LO phase relationship must be known in addition to being constant
      All differential phase delays in the system (cable lengths, 
amplifer phase shift, etc.)  must be known, often by careful matching 
and/or calibration

The MIMO issues are all taken care of by the motherboard (USRP1 or 
USRP2) so long as the daughterboards all lock their local oscillators to 
the motherboard clock.  All of them, except for the TVRX, do this.

> A. Output buffers either need to be flushed to start filling when the trigger 
> occurs, or the next sample after the trigger needs to be marked in some fashion 
> so that it is identified in the data stream(s).
>   

This is done implicitly.  All samples from all antennas are time-stamped 
so you know which ones from each antenna correspond to each other.

> B. The state of the decimation counters for each USRP2 needs to be either reset 
> or read when triggered.  An alternative is to count the number of 100MHz clocks 
> before the next internal sample is output to the ethernet buffer.  Not 
> compensating for this could lead to a large time delay variation betweeen 
> channels depending on the decimation factor.  This is translated to a phase 
> shift at the center frequency.  For example, we like to run narrow-band outputs 
> with a sample rate of 50kHz.  This is a decimation of 2,000 relative to the 
> 100MHz clock.  With a 30MHz center frequency input each sample is 30/100, or 
> 3/10 of a cycle so a decimation difference of 2,000 clocks (20 microseconds) is 
> as much as 600 cycles of the RF signals.
>   

This is how we do it.  All decimation and interpolation counters (and 
everything else in the signal processing chain) are triggered by the 
master system timer, which is coordinated between systems.

> C. The phase of the internal numerical oscillator needs to be synchronized 
> between USRP2 systems.  This would normally be through a common, synchronized 
> reset, but could also be done by reading the phase register for the numerical 
> oscillators when the trigger/1pps occurs.   This could lead to as much as a 360 
> degree phase variation if not compensated.
>   

This is also done.

> D. The 1PPS/trigger input needs to be resynchronized internal to the USRP2 so 
> that it is stable with respect to 100MHz clock edges. This implies meeting 
> setup and hold times for the 100MHz clock edges.  Since the 100MHz clock edges 
> are not observable (or are they?) external to the USRP2 the 1pps needs to be 
> synchronized to the 10MHz reference clock and then internal to the USRP2 
> resynchronized to the 100MHz such that it is stable with respect to the 10MHz.  
> It needs to meet 100MHz setup and hold times internal to the FPGA.  This could 
> lead to a channel-to-channel phase variation of 3/10 of a cycle at 30MHz, 
> sufficient that beams created from the multiple channels are not formed 
> properly with potentially significant loss in gain and high sidelobes.
>   

In the case of a MIMO setup over the MIMO cables, this is handled 
internally by the FPGA, since all signals are generated from the same 
100 MHz clock.  In the case of "Scenario 3" from my previous post, in 
which every system in synced by the 1 PPS and the 10 MHz reference, the 
1 PPS input reference signal is required to be synchronous with the 10 
MHz reference signal.  This allows each USRP to sample it with its own 
100 MHz clock without any setup/hold issues.  Remember, the all the 100 
MHz signals are phase locked to the same 10 MHz reference.  Thus, each 
system has the same conception of time.

Note that not all of the common GPS disciplined oscillators out there 
produce a 1 PPS signal which is synchronous to the 10 MHz signal.  
Nearly all do, but the CNS Systems CNS Clock II does not, so it would 
not be a good candidate to use.

> A through C could be straightforward through a master synchronized reset or by 
> including metadata with the data block.
> D is not straightforward since it requires an analysis of the timing internal 
> to the USRP2.  The ability to implement either item D or the master 
> synchronized reset will not be clear until the schematic is published.  With a 
> 10MHz external clock input I expect the best we will be able to do for 
> channel-to-channel coherence is to align triggers to the 10MHz clock edge for 
> resolution of 100nsec.
>   

The schematics are all published on the Wiki (see the USRP2 FAQ). 

I assure you that we always get our 100 MHz cycles lined up properly.


> Without these synchronizations the apparent phase or time delay will wander 
> even for a single channel of normally synchronous data from one trigger to 
> another.
>   

I would say that all of the above, A through D are required in order to 
call it MIMO, and I would not have designed the system any other way.  
That being said, there is still some work to do inside the FPGA, 
firmware, and host software.

> When you are able to make the schematic available I would be happy to help 
> develop the multi-channel synchronization approach.  My preference is your 
> scenario 3 since it is the most scaleable.
>   

That is great!  The schematics are on the wiki at 
http://gnuradio.org/trac/wiki/USRP2

Please let me know if you have further questions or want to discuss 
anything in more detail.

Matt