I’ve just solved this problem in a following
way: I increased the mag rate correction coefficient from 2.5 to 50. Seems to
have quicker response.
The magnetometer should only be needed to compensate for drift in the
attitude integrated from the gyros, so if your response is slow or you need a higher correction coefficient, it
probably means that your gyro calibration (GYRO_x_SENS) is not good.
Did you check this?
You can also disable the mag via <configure name="USE_MAGNETOMETER" value="0"/>, use the real-time plotter to plot roll,pitch and yaw. Then yaw the quad in 90deg steps or so and you should see that nicely in the plotter... if you get less than that it indicates that your gyro sens is too low.
But there is next
problem – the 2d mag calculation works not really perfect while banking.
The rotorcraft has now less stability while braking in flight. But anyway it’s
better now.