Cortex-M4F FPU problems
Why don't you try to remove all the floating point calculations from the interrupt service routines. Many folks would point out that it shouldn't be in there in the first place. 

Pre-calculate floating point things in the mainline code flow so that it does not need to be in the ISR. This may require some rethinking of the algorithms used but will often lead to more robust and leaner code.