Is there a way to limit the number of CPU Cores used for physics baking?
There isn't really a straight forward answer for this, it will depend on what simulation you want to bake and what OS you are using.

To use multiple cpus, some parts of blender use openmp and others use pthreads.

For simulations using openMP, setting the OMP_NUM_THREADS environment variable should limit the cpus used.

For simulations using pthreads, adjusting system limits should be able to limit the cpus or cpu time used. The way to do this varies between most operating systems, for linux there are /proc/... settings or the bash builtin ulimit command.

Another approach available on most *nix systems is the nice and renice commands to adjust the cpu priority so that other procs are given cpu time before blender.