Re: [Discuss-gnuradio] Constant carrier digital transmission

[Ed Criscuolo]

In a GnuRadio block that is derived from a "sync" block, the number of
input and output items are the same, so the work function is supplied
with a single value called "nitems".  The normal control flow of a
typical work function uses a loop that iterates over nitems, taking
one item from the input(s) buffer(s) and producing one item into the
output buffer. This process repeats until all input items are consumed
and all output buffer item spaces are full.  The work function then
returns a value of "nitems" to indicate to the scheduler how many items
were consumed and produced.

But it is perfectly legal for the loop in the work function to terminate 
early, in which case it should return the number of items
_actually_ processed.  The scheduler will take care of keeping track
of the partial buffers and no data will be lost.  But performance
will be reduced.

GnuRadio blocks are timing agnostic.  They run as fast as they can,
and process an entire buffer's worth of items (typically 64K items)
per call to the work function.  This is done to improve throughput.
Timing only happens when your bitstream meets up with a hardware
device, like a USRP. But as long as the _average_ throughput is
fast enough to keep up with the hardware, everything is fine.
The amount of CPU processing "headroom" determines how far you can
degrade a block's average throughput without creating underruns.

This level of understanding is fundamental to writing any GnuRadio
blocks, so you really should take the time to play with it and get
to know it.

Here is a snippet of VERY OLD code for example only. It that always
runs "nitems" times. When idle (ie - no hdlc packets available), it
produces 64k bits (8k of flags) on every invocation! This results in a 
large latency when a packet finally does come along.
It should be modified to break the loop on the second fifo empty
test and return "i" instead of "nitems"

int
sc_hdlc_router_source_b::work (int                       noutput_items,
                                       gr_vector_const_void_star &input_items,
                                       gr_vector_void_star       &output_items)
{
  unsigned char * outbuf = (unsigned char *) output_items[0];
  int             i;

  // Loop until the requested number of output stream bytes have been 
generated
  for(i=0; i<noutput_items; i++)
  {
    if (d_fifo.empty())
      {
        // Process a single incoming packet on the tun/tap pseudo 
network device
        // by bitstuffing it, encapsulating it in an MPoFR frame, and 
pushing
        // it into the bit_buf fifo.
        encapsulate_incoming_packet();
      }

    // If bit_buf fifo is still empty, push a FLAG into it
    if(d_fifo.empty())
      {
        push_flag();
      }

    // Output a bit from the bitbuf in the low-order bit of the output byte
    outbuf[i] = d_fifo.pop();
  }

  return noutput_items;
}


@(^.^)@  Ed


On 8/16/16 8:12 PM, Inspire Me wrote:
> Hi Ed
>
> Thank you for the response.
> I like the idea of the second option. Could please expand the
> explaination. I am new to gnuradio and don't yet fully grasp how the
> scheduler works in detail. How do you terminate the work function early
> and ensure the single 0x7E gets streamed immediately to the modulator so
> that we don't get under runs. How quickly will the work function be re
> executed by the scheduler ? As fas as CPU is concerned, I have a machine
> with 16 cores ( 32 threads). I am not concened if one thread is consumed
> by this. Could you send me a snippet of the work function you implemented.
>
> Your assitance is much appreciated.
>
> Kind Regards
> Frank
>
> On 17 August 2016 at 02:00, Ed Criscuolo <address@hidden
> <mailto:address@hidden>> wrote:
>
>     Hi Frank,
>
>     I'm the author of the GMSK Spacecraft Groundstation project you
>     referenced.  Been on vacation for the last 2 weeks without internet,
>     so I'm just seeing this.
>
>     All in all, Marcus is giving you good advice.  At the time I wrote
>     it, there were no tags or messages, so everything related to
>     PDU-like behavior had to be implemented in a single  C++ block. Now,
>     packets can come in as PDUs, and get converted into a bitstream in a
>     single, much simpler block.
>
>     As for the issue of putting HDLC flag octets out back-to-back as the
>     "idle" pattern between frames, I had tried two different solutions,
>     one of which Marcus touched upon.
>
>     First approach was to simply reduce the buffer size of all blocks by
>     recompiling GnuRadio with the appropriate global constant reset to a
>     smaller value.  This has significant impact on overall performance,
>     but because our data rate was modest (only a few hundred Kbits/sec)
>     and we had some wicked fast hardware (for the times!) we were able
>     to get this to work, reducing the added latency of outbound packets
>     from over ten seconds to only a few seconds. This was OK for our
>     particular requirements, but not for a general purpose system.  It
>     was an easy but VERY brute-force solution.
>
>     Later, I modified the block to to implement a solution similar to
>     Marcus' "Option A".  Every time the block's work function was entered,
>     it checked for an available packet (I was using sockets, but modern
>     GnuRadio would use PDUs and message queues). If a packet was available,
>     it would be bitstuffed and the bitstream written to the output
>     buffer. This would continue until either no more packets were
>     available, or the
>     output buffer reached capacity.  If there were NO packets available
>     when the work function was called, it would put a SINGLE 0x7E (that
>     is, a bitstream of eight bits) on the output buffer and terminate
>     the work function early, returning only the numbers of bits actually
>     produced.
>     This is perfectly allowable, but it will result in the block consuming
>     a large amount of CPU when generating idle flags.  At least this
>     solution only effects a single block instead of all the blocks.
>
>     If the resulting CPU usage is too high, it could be reduced, at the
>     cost of adding latency, by outputting 2 or more flags at a time.
>
>     Hope this helps.
>
>     @(^.^)@  Ed
>
>
>     On 8/15/16 7:00 PM, Inspire Me wrote:
>
>         Hi Nick / Marcus
>
>         Great discussion, obviously this issue has been solved before, as
>         mentioned in the 2009 discussion the GMSKSpacecraftGroundstation
>         project
>         (
>         https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation
>         
> <https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation>
>         <https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation
>         
> <https://moo.cmcl.cs.cmu.edu/trac/cgran/wiki/GMSKSpacecraftGroundstation>>
>         )
>         seems to have solved it. I acknowledge that gnuradio has moved
>         on, but I
>         would like to see how it was solved and possibly adapt it to the
>         current
>         gnuradio. *Is there anyway to get access to the code ?* I am
>         stuck at
>         the moment.
>
>         The only idea I have is to impelement a source block with very small
>         buffers to feed into the Tx Chain and some sort of switch to toggle
>         between data stream and flag stream based on a message / tag
>         from up the
>         chain.
>
>         Nick can you elaborate on the modification of the Work() function. I
>         thought this was scheduled only when sufficient data is in the Input
>         buffer ? Is there a way to have this execute more often or when
>         no data
>         exists in the input buffer ? Is there a way to implement a
>         switch block
>         that monitors input streams and efectively switches the output
>         stream
>         such that it is not blocked waiting for input from one of the
>         streams ?
>
>         Once again thank you for the help
>
>         Kind Regards
>         Frank
>
>         On 16 August 2016 at 03:14, Marcus Müller
>         <address@hidden <mailto:address@hidden>
>         <mailto:address@hidden
>         <mailto:address@hidden>>> wrote:
>
>              Hi Nick,
>
>
>              On 15.08.2016 19:19, Nick Foster wrote:
>
>                  This is a pretty familiar problem. A lot of satcom
>             systems require
>                  continuous transmission when in an idle state so the
>             receiver can
>                  use slow (i.e., sensitive) frequency and timing
>             estimator loops.
>                  I'm sorry to say there's no great answer.
>
>                  But you have some options:
>                  a) modify the HDLC block so that when work() is called,
>             if nothing
>                  is in the msg queue just output some fixed number of
>             0x7Es and sit
>                  on its thumbs until the next call.
>
>              'doh! Could've thought of that; makes a lot of sense, here.
>
>                  b) modify whatever block is generating messages so that
>             it outputs
>                  0x7E frames periodically
>                  c) modify the FPGA (or whatever DUC backend you're
>             using) to both
>                  do the modulating and send 0x7Es (modulated) when
>             there's nothing
>                  coming in
>
>                  Each approach has its disadvantages:
>                  a) because Gnuradio buffers can be quite large, you
>             will incur a
>                  latency penalty as GR (i.e., all of the buffers of all
>             of the
>                  blocks downstream of the HDLC framer) propagates your
>             frame downstream
>
>              Not if he adds the functionality to include a tx_time tag; that
>              wouldn't /reduce/ latency, but it would allow for time-slotted
>              transmissions!
>
>                  b) message sources don't have backpressure, so this
>             won't really
>                  work -- either you'll underflow and interrupt your
>             sample stream
>                  or build up a huge stack of unsent data just as in a)
>                  c) this is a lot of work, and requires that the whole
>             modulator,
>                  soup-to-nuts, is implemented in the FPGA.
>
>                  I'd start with approach a) and work your way towards c)
>             if your
>                  application can't handle the latency.
>
>                  --n
>
>              Best regards,
>              Marcus
>
>              _______________________________________________
>              Discuss-gnuradio mailing list
>         address@hidden <mailto:address@hidden>
>         <mailto:address@hidden <mailto:address@hidden>>
>         https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>         <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>
>              <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>         <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>>
>
>
>
>
>         _______________________________________________
>         Discuss-gnuradio mailing list
>         address@hidden <mailto:address@hidden>
>         https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
>         <https://lists.gnu.org/mailman/listinfo/discuss-gnuradio>