paparazzi-devel
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Paparazzi-devel] Libopencm3 - STM32-F1 change interrupt priority

From: Felix Ruess
Subject: Re: [Paparazzi-devel] Libopencm3 - STM32-F1 change interrupt priority
Date: Sat, 24 Aug 2013 00:01:26 +0200
Hi Michal,

well... the sys_mon module shows you only part of the whole deal... and it was written before we split up the main_periodic into serveral separate periodic tasks and the naming of periodic_time and periodic_cycle doesn't quite fit anymore

Maybe we can come up with better statistics/heuristicsa and hence also a better cpu_load estimate. Right now it is simply:
  sys_mon.cpu_load = 100 * sys_mon.periodic_cycle / sys_mon.periodic_time;
Where the periodic_cycle is:
  periodic_cycle = periodic_usec - n_event * min_time_event;
meaning we measure the minimum time spent in one event loop (min_time_event) and assume that is is how much time it takes to just check all events/flags without actually having any new event and acting on them.
Then we multiply the min_time_event by the number of times the event loop was executed in the last second (n_event) giving us a very very rough estimate of the time it's "idling".
Subtract that from the time between two periodic calls (periodic_time_usec) to get the time spent in the period functions.

Ideas on how to improve this are welcome!

In the master branch I just added min/max for the periodic time to it as well, as at this point this is actually more relevant.
So we at least have a bit more information...

I also observe quite some jitter in the periodic_time
This has several reasons:
  • The periodic functions are checked from the virtual sys_time timer (you can have lots of them...)
    It runs at SYS_TIME_FREQUENCY, which defaults to two times the PERIODIC_FREQUENCY.
    So the resolution of it is rather coarse. You can set a higher frequency, e.g. <define name="SYS_TIME_FREQUENCY" value="2048"/>
  • It is still a cooperative scheduling scheme and a really simple one without priorities (except of hardware stuff like DMA transfers, etc.).
    So it basically relies on the assumption that each event cycle (and also the other periodic tasks) don't take too long.
So we need to take a closer look and think of simple ways on how to improve the "sheduling" (shy of using a real RTOS scheduler with priorities, which we'll hopefully get soon).

So much for now...

Cheers, Felix
    


On Fri, Aug 23, 2013 at 8:19 PM, Michal Podhradsky <address@hidden> wrote:
Hi Felix,

thanks for the explanation about the sys_mon, now it makes more sense:)

I ran a couple of quick tests (just looking at the telemetry data and sys_mon module, no data logging):
A) rotorcraft running @512Hz + GX3 as ahrs @500Hz + int_euler for stabilization -> runs just fine (master 5.0)
B) rotorcraft running @512Hz + GX3 as ahrs @500Hz + float_euler for stabilization -> VERY BAD timing (probably because of the floating point calculations at high rate?) (master 4.9)
C) rotorcraft running @512Hz + GX3 as ahrs @125Hz + float_euler for stabilization (master 4.9) -> OK(!), note that floating point calculations runs at the same rate as in B), there is just 4x more UART traffic...

Maybe rotorcraft code is written in a way it can handle the tasks and timing better?

M

As for yesterday tests w. fixedwing, here is the configuration for aspirin:
  <firmware name="fixedwing">
    <target name="ap"             board="lisa_m_2.0"/>
    <configure name="FLASH_MODE" value="JTAG"/>

    <define name="AHRS_USE_GPS_HEADING"/>
    <configure name="PERIODIC_FREQUENCY" value="500"/>
    <define name="MODULES_FREQUENCY" value="500"/>

    <subsystem name="control"/>

<!-- Communication -->
    <subsystem name="telemetry"     type="transparent"/>

    <subsystem name="radio_control" type="ppm">
       <configure name="RADIO_CONTROL_PPM_PIN" value="UART1_RX"/>
    </subsystem>

<!-- Sensors -->
    <subsystem name="imu"      type="aspirin_v2.2"/>
    <subsystem name="ahrs"      type="int_cmpl_quat"/>
    <subsystem name="gps"       type="ublox"/>
    <subsystem name="navigation" type="extra"/>
    <subsystem name="ins" type="alt_float" />
  </firmware>

<!-- Modules -->
  <!--modules main_freq="500"-->
  <modules>
    <!-- Airspeed Sensor -->
    <load name="airspeed_adc_adv.xml">
    </load>

    <!-- System Monitor -->
    <load name="sys_mon.xml"/>
    <load name="battery_monitor.xml"/>

    <!-- UGEAR BLACKBOX -->
    <load name="ugear_blackbox_uart.xml"/>
  </modules>


On Fri, Aug 23, 2013 at 8:15 AM, Felix Ruess <address@hidden> wrote:
Hi Michal,

I added some more information to http://paparazzi.enac.fr/wiki/Module/System_monitor. I hope that answers your question about the periodic_cycle...

Nice job on the testing/plotting so far ;-) We clearly need to have a closer look at this...
I didn't observe any variations like that in my setups so far (will check again tonight when I have access to the hardware again).
There are quite a number of reasons why this could happen...
E.g. if have an algorithm that takes a lot of time, it could delay the next periodic function. As we don't have any preemption (yet) this is some sort of cooperative scheduling.
It could be triggered by an event like new gyro data and hence run in the event loop, or be one of the autopilot periodic functions like running guidance (e.g. if you try to run that in float without a floating point unit it will probably take too long)

What configuration (stabilization, ahrs, ins, etc..) were you using for the aspirin tests?

Cheers, Felix


On Fri, Aug 23, 2013 at 2:50 AM, Michal Podhradsky <address@hidden> wrote:
Hi Felix,

so I looked into it a bit more. First test with system monitor, using fixedwing with GX3 at 500Hz, and PERIODIC_FREQUENCY=500Hz.
The periodic time is an average, so it is exactly at 2ms. The max periodic cycle is however over the periodic time -> something is going on there.

Although I am not sure how to interpret the periodic_cycle. The average is 1.25ms, which represents 800Hz, but I am not aware of any loop going that fast.

I made a simple logger module to monitor system time on the autopilot. It runs at the same frequency as the periodic frequency, and sends through UART current system time  (get_sys_time_usec() from arch/stm32/mcu_periph/sys_time_arch.h). I ran couple of tests with different configurations.

Test 1: -> data1.png
Aspirin IMU, 60Hz periodic frequency. Just to get the baseline.
After first 80sec I plugged in GX3 configured to just output data @500Hz (loads of data on UART). The data from GX3 weren't processed any further though.

I calculated difference between the received timestamps and converted that to frequency. Data1 shows the result. You can see that most of the data come with frequency between 58 and 62Hz, which is +-3% around the defined periodic frequency. It is not great, but sufficient for now. Then every half second you get a timestamp at 56.6/64Hz (that is +-6% which is not acceptable precision).

At T=~80sec I plugged in GX3 and you can see that the frequency of timestamps starts to oscillate. Not much, but it definitely observable.


Test 2: -> data2.png
GX3, 60Hz periodic frequency. GX3 configured to output attitude data @125Hz which is slightly higher, but its only effect is a higher load on CPU. See data2
Most of the time the measured frequency is right 60Hz, but every 250ms the loop gets delayed (you can see the frequency drops to 58Hz and then jumps to 62Hz (probably the next loop is somehow faster).

So at lower frequencies everything works just fine.


Test3 -> data3a.png and data3b.png
GX3, 500Hz periodic frequency, GX3 @500Hz.

Now the things starts to be messy. You can see that the system frequency is all over the place, and there is some periodicity in the behaviour. The mean is still 500Hz, but the actual frequency is between 400Hz and 600Hz.

Test4 -> data4.png
Aspirin @500Hz, basically the same behaviour as with GX3.

Since the problem with timing is the same with Aspirin as with GX3, it probably wont be UART/SPI drivers/isr routines.

Any idea which other part of code could delay the loop like that?

Cheers
M


On Wed, Aug 21, 2013 at 2:34 AM, Felix Ruess <address@hidden> wrote:
Hi Michal,

That's rather bad... I don't have any timing problems here, but then I'm not blowing heaps of data through UART...
You can also use the sys_mon module [1] to check if the timing of your main periodic is ok.
The priorities should probably be configurable.... can you make some tests if this helps or if it is something else?


Cheers, Felix


On Wed, Aug 21, 2013 at 1:02 AM, Michal Podhradsky <address@hidden> wrote:
Hi all,

I have a question about timing on Lia 1.1 (stm32f1). I recently noticed that the timing on the MCU is not solid.
I made a simple module with a periodic function which toggles an LED, ran it at PERIODIC_FREQUENCY and then plugged in a scope to see if the LED switching has a constant frequency.

I am using PERIODIC_FREQUENCY=512Hz, and the frequency moves around 30%, and occasionally bounces down to ~300Hz. If necessary, I can post video from the scope if needed.

Seems to be caused by interrupt priorities (e.g. excessive traffic on one of the UARTS can delay the main loop etc). The function nvic_set_priority() which mentioned Esden is used (master) only on a few peripherals (i2c, spi, adc, ppm), most often setting the priority to 0.

According to Cortex M3 programming manual, "If software does not configure any priorities, then all exceptions with a configurable priority have a priority of 0."
if all the configurable interrupts have the same (highest) priority, then there is no way to move the more important interrupts (e.g. UART with lots of data coming through) any higher.

The possible solution seems to be to set explicitly isr priority on each peripheral (maybe an issue for github?).

What do you think of that? Did you have similar issues with the timing?

Regards
M



On Mon, Apr 22, 2013 at 10:14 AM, Michal Podhradsky <address@hidden> wrote:
Hi Esden,

thanks for clarification!

M


On Fri, Apr 19, 2013 at 4:01 PM, Piotr Esden-Tempski <address@hidden> wrote:
Hi Michal,

I think you are mistaking the vector table slot numbers there for priorities. These are the positions of the function pointers inside the vector table. If you change those numbers then things will obviously explode.

(just for reference, here the defines get their weak functions assigned: sw/ext/libopencm3/lib/stm32/f1/vector_nvic.c, and here the vector is being put together: sw/ext/libopencm3/lib/cm3/vector.c)

IRQ priorities are set using the NVIC_IPR register, or using nvic_set_priority function.

I hope this helps.

Cheers Esden

P.S. both files nvic.h and vector_nvic.c are generated using the sw/ext/libopencm3/scripts/irq2nvic_h from sw/ext/libopencm3/include/libopencm3/stm32/f1/irq.yaml so you should not edit those files by hand.

On Apr 19, 2013, at 2:12 PM, Michal Podhradsky <address@hidden> wrote:

> Hi folks,
>
> I have a question about interrupt priorities for STM32F1 chip (Lia 1.1/Lisa_M 2.0).
>
> In sw/ext/libopencm3/include/libopencm3/stm32/f1/nvic.h are defined priorities for user interrupts. However, if I try to change the priority for example for NVIC_USART2_IRQ (let's say make it higher priority than NVIC_USART1_IRQ), the  code compiles, but then the program hangs up instantly in usart_isr interrupt routine (debugged with JTAG).
>
> Can the priorities be set somewhere else or is it a feature to have "hardcoded" priorities?
>
> Thanks
> Michal
> _______________________________________________
> Paparazzi-devel mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/paparazzi-devel


_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel



_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel



_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel



_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel



_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel



_______________________________________________
Paparazzi-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/paparazzi-devel


reply via email to
[Prev in Thread] Current Thread [Next in Thread]