Disabled constraints still being calculated?
It's not the constraint that's the cause. The common way to deal with this is to enable "simplify" and globally reduce the subdivision level while you're animating.