When optimizing the MSE, the correlation between prediction and target increases?
It should not be lower but it does not always have to be higher.

Let's consider these two vectors:

A | B
1 | 2
2 | 3
3 | 4


The correlation between A &amp; B is super high as I've just modeled a deterministic relationship (B = A +1). However if I perceive B as a predicted value and A as the real value then the MSE would not be 0.

Now let's say I improve my prediction and it's perfect:

A | B
1 | 1
2 | 2
3 | 3


We haven't impacted the correlation between those vectors at all because we simply removed the constant shift between the vectors. Their relation is still 1:1, however the MSE if B is the prediction and A is the real value is now much, much better than in the first example.