Re: [Discuss-gnuradio] SCHED_FIFO and NPTL

[Eric Blossom]

On Wed, Mar 08, 2006 at 02:18:35AM -0500, Frank Brickle wrote:
> Eric Blossom wrote:
> 
> >Using LD_ASSUME_KERNEL=2.4.19 effectively forces the old (pre-NPTL)
> >behavior, which means that acquiring an uncontested mutex requires a
> >trip to the kernel.  I believe it also means that mutexes won't work
> >in shared memory across process boundaries.  Those seem like total
> >losers to me...
> 
> This adds up to: futexes don't work (yet). Is that right?

I guess that's the real question.  

My understanding is that they work, it's just a question of how they
work vis-a-vis real time.  In our case I'm not worried about issues
such as priority inversion on mutexes.  I assume all our signal
processing threads will be running at the same pri and waking up
anybody who's waiting will be fine.  I just want to ensure that I run
before the X-server.

> I'm still confused about a couple of things. On one hand the holy grail 
> appears to be getting the number of frames in a jack buffer down to 64 
> so as to minimize the roundtrip latency. On the other hand you want to 
> eliminate xruns. They aren't the same by any means.

Definitely.  In a hard-real-time world, this problem is solvable.

At this point, I've got my eye more on minimizing the USRP latency
since it determines the tightest MAC loop we can build on the host.
Basically, I want to make sure that I'm running at a better pri than
the X-server and friends.  (This all assumes that I've got sufficient
cycles to actually get the work done in real time.)

The smallest chunk of work we can conveniently get across the USB is
128 complex samples (512 bytes).  Assuming we're running at 4MS/s
that's 32uS of samples.  If we're able to get N uS of CPU every
32uS, then solving the audio problem shouldn't be a problem ;)

(I'm also reviewing chapter 5 of the USB 2.0 spec to see how often
the h/w arbitrates for access to the USB.)

> It's not clear that minimizing roundtrip latency means much when you're 
> using DSP buffers of 512 frames or more. By the same token, in what I've 
> observed, the chief culprit for the xruns is the X window system. There 
> is a very delicate balancing act going on in the process priorities 
> between the audio subsystem and the video subsystem. I'm not convinced 
> that running SCHED_FIFO is going to routinely enable the audio subsystem 
> to slide in under the video action under all circumstances.

Taking a look at the output of 

  $ ps -eo pid,tid,class,rtprio,ni,pri,comm 

on my system indicates that the X-server is not running with real time
scheduling.  Only the migration tasks are.  I assume these migrate
tasks across CPUs (this output is from a dual core machine).  The
stuff with negative niceness (the NI column) also have preferential
scheduling over other time shared processes, but should be shutout by
the SCHED_FIFO stuff.

  PID   TID CLS RTPRIO  NI PRI COMMAND
    1     1 TS       -   0  23 init
    2     2 FF      99   - 139 migration/0
    3     3 TS       -  19   5 ksoftirqd/0
    4     4 FF      99   - 139 migration/1
    5     5 TS       -  19   5 ksoftirqd/1
    6     6 TS       -  -5  29 events/0
    7     7 TS       -  -5  29 events/1
    8     8 TS       -  -5  29 khelper
    9     9 TS       -  -5  29 kthread
   20    20 TS       -  -5  29 kacpid
  619   619 TS       -  -5  29 kblockd/0
  620   620 TS       -  -5  29 kblockd/1
  667   667 TS       -   0  24 pdflush
  668   668 TS       -   0  24 pdflush
  669   669 TS       -   0  24 kswapd0
  670   670 TS       -  -5  28 aio/0
  671   671 TS       -  -5  28 aio/1
 1273  1273 TS       -  -5  29 kseriod
 1327  1327 TS       -   0  24 kirqd
 1391  1391 TS       -  -5  29 ata/0
 1392  1392 TS       -  -5  29 ata/1
 1402  1402 TS       -   0  14 scsi_eh_0
 1427  1427 TS       -   0  24 scsi_eh_1
 1515  1515 TS       -  -5  29 reiserfs/0
 1516  1516 TS       -  -5  29 reiserfs/1
 2560  2560 TS       -  -4  27 udevd
 3151  3151 TS       -   0  19 shpchpd_event
 3181  3181 TS       -  -5  29 khubd
 3239  3239 TS       -  -5  25 hda_codec/0
 3240  3240 TS       -  -5  25 hda_codec/1
 3816  3816 TS       -   0  24 khpsbpkt
 3824  3824 TS       -   0  24 knodemgrd_0
 4068  4068 TS       -   0  20 pccardd
 5921  5921 TS       -  -5  29 ipw3945/0
 5922  5922 TS       -  -5  29 ipw3945/1
 5923  5923 TS       -  -5  29 ipw3945/0
 5924  5924 TS       -  -5  29 ipw3945/1
 6714  6714 TS       -  -2  25 ipw3945d
 6719  6719 TS       -   0  23 syslog-ng
 6722  6722 TS       -   0  24 klogd
 6799  6799 TS       -   0  23 resmgrd
 6875  6875 TS       -   0  23 dbus-daemon

[snip]
 
 8125  8125 TS       -   0  23 kdm
 8149  8149 TS       -   0  24 X             <<< --- X-server
 8151  8151 TS       -   0  23 cupsd
 8156  8156 TS       -   0  24 kdm


> Bottom line, it hasn't actually been proved that running SCHED_FIFO will 
> squash the existing latency and continuity problems, so I'm not at all 
> sure much is missing without that capability.

Frank, is this a statement or a question?

I'm not sure if the JACK FAQ is up to date or not.  Do you have data
on the success or failure of folks running JACK on ALSA using NPTL and
if they are able to get sufficiently good performance?  I guess Ardour
users would be a good test case.  How about the dttsp stuff under Linux?

Returning to the audio xrun problem: I think that in the typical USRP
+ audio situation, xruns are aggravated by differences in the clocks
between the two domains and the fact that we aren't doing anything to
handle that situation.

Eric