WFI instruction slowing down SYStick interrupt
I'm not familiar with these devices, but in the user manual it is mentioned that:


  The Sleep state of the system corresponds to the Sleep state of the
  CPU. The state is entered via WFI or WFE instruction of the CPU. In
  this state the clock to the CPU is stopped. The source of the system
  clock may be altered. Peripherals clocks are gated according to the 
  SLEEPCR register.


Taking a look at the SLEEPCR register description shows:

Field  Bits Type Description 
SYSSEL 0    rwh  System Clock Selection Value
                 0 -&gt; f OFI clock, 1 -&gt; f PLL clock


as the reset value of said register is 0, it defaults to the \$f_{OFI}\$ clock. Which runs at 36.5 MHz uncalibrated or 24 MHz calibrated. It has a rather huge accuracy range if uncalibrated (see the datasheet) of at least +-15%, together with the fact that you will get some systicks with 120 MHz in again during your execution it might well turn out to be roughly 40MHz in average.

Solution (hopefully): Set the SYSSEL bit to 1.