Multi threaded CPU intensive task throttles CPU way before temperatures limits
This may be a longshot, but perhaps the difference in single-core performance and/or cache performance between the 2012 and 2016 cpu packages are large enough that the cores are data-starved and throttling down until they are able to work again? 

I'm making that guess because you indicate enough single-thread processes can run full speed on all cores, and a simple multi-thread program can run full speed on all cores.

That makes me think there is something in the program design of your real workload vs the test multi-thread workload that isn't letting the CPUs work all the time