Atmega328 fast PWM (mode 7) of timer 0 not working as expected
In your second setting the timer is incrementing once for every clock cycle (prescaler is 1), but it only counts to 16. The AVR executes one instruction per clock cycle (except for branches) so there are less than 16 instructions available between each PWM interrupt.

The execution time of the ISR is longer than this (it may be variable as well) and that is causing your code to periodically miss interrupts, extending the apparent time it is taking to count to your target value. To fix the problem you need to either use the prescaler and count to a higher value, or if you can't do that (if you are using the hardware PWM generator), look at using OCR1B with a longer count to generate the interrupts instead.

Also you should disable interrupts when you clear timerCount because this is a 32-bit value that requires multiple instructions to update on an 8-bit platform.