Why does baking a fluid sim use so little CPU?
There are two contributing factors. One is that the code for the fluid sim has not been rewritten to support take advantage of multi-threading. This means that only one, discreet process will be running.

The other factor is task-switching. Windows has literally hundreds of processes that need CPU time at any given moment. Each of these is queued for time on a processor. One will get switched onto the processor for a few milliseconds, and then it will be moved to the back of the line (sort of) to wait for its next turn. You have 100% of one process that's spending a quarter of its time on each core... but not simultaneously.

Now for the LONG answer...



An excellent metaphor is checking out at a grocery store.

The one special rule at this grocery store is that you can only pay for one item at a time. You pay for one item and then you go back to the line to wait to pay for another item. You keep going to the back of the line until you don't have any more items to pay for, which would be like when a program exits.

With a multi-core processor, you have one line, but several checkers. You go to whichever checker is available when your turn comes up. Blender is checking out a lot of items, but it's not limited to one lane, so it can go to any of the four checkers. Blender is showing 25% on each core because its one process is doing 100% of its work, split between 4 cores.

This is what happens when one program has one batch of things to do on the CPU.

But what if a program had a partner who could also check out some of the things, too? If a program could copy itself, it could give half of its batch of things to the copy and check out twice as often. This is what is meant by writing a program for multi-threading. It has explicitly have (create) a partner (thread).

If there's a partner, and if your batch of things can be split into two completely separate batches, then you can have two people waiting in line, each with half of the things. They get done twice as fast. 

This is why GPU processing is so fast. It's streamlined for situations where you have 9000 of exactly the same kind of item to check out. The GPU has (random guess) 9000 check out lines. It will give 1 item to 9000 people, and check them all out at the same time. Done.

The trick is, the separate threads have to be (almost) completely separate programs. You can't split a ladder in half, and pay for half of it at one checker and half at the other. You also can't take two numbers that you want to add together and do one of them on one processor, and the other on another processor.

They can't share variables, they can't share memory space (sort of), they can't communicate with each other the way that different functions/methods do within the same program. This means if they're going to multi-thread, then the process has to be non-linear.

For example, if you and I were going to cooperate on adding a bunch of numbers, that'd be great. You add half of them, I'll add the other half, and we'll add our results together. We don't have to be synchronized, and neither of us has to know what the other is doing. We just add whatever the results are, and we'd get done in half the time.

Back to your original question, if you and I were going to cooperate on calculating where a particular molecule of water was during a particular second, we'd get almost no benefit over just one of us doing it. You'd have to wait for me to calculate a current position, before you could even begin calculating the next position. Then I'd have to wait on you, back and forth.

Some processes aren't suited to multi-threading. Other processes just have to find the right factor to split on, so that things can be done in parallel.