How to improve peltier efficiency, use a CPU cooler?
If you attach the CPU cooler to the heatsink on the black Peltier box, then it will help cool the water but probably only by a very small amount.

There is a limit to the amount of cooling the Peltiers can provide.  They essentially pump heat from the input side to the output side, and they are a little more efficient when the output side is cooler.  If the output side heatsinks (the black finned bits on the the Peltier module) are hotter than air temperature, then cooling them down will improve the efficiency.

The reason I don't expect it to work very well is because  there is already a large fan-cooled heatsink present. The CPU cooler adds some more heatsink, but not much relative to what is already there.

If you want to get a better idea of how well this could work, then you could measure the temperature of 1) the air, 2) the heatsink on the module, 3) the water as it leaves the module, 4) the water as it comes back.  This would let us calculate which section (matrix, peltier, heatsink) was the bottleneck and how to improve it.  The CPU cooler would only be effective if the heatsink was the bottleneck, but my money would be on the peltier being the bottleneck, as it is the most expensive part.