Neural network regression is not dynamic enough to predict target range?
I've solved it myself. Actually the whole premise was incorrect. It was tempting to look at the plot and think that it just takes a linear transformation to change the slope of the line, but that's because one axis of the plot is the target data!
My claim was that the model should stretch the data along the y-axis (around the mean) because it's the obvious thing to do. But is it really? Have a look at these 4 examples. According to my &quot;obvious fudge factor&quot;, the two points on the right should have been stretched up, and the two on the left should have been stretched down.

But then what would that mean for all the other points that were on target? It would shift them off target and overall the model would perform worse. Here's what would happen to some of the points stretched up away from the mean.

The tempting human thing to do, which is what got me, is to include the x-axis in your perception of the problem. But we have to remember that it's not there for the model.
Every time the model predicts deviations from the mean it needs to have a pretty strong reason to do it. If it hasn't actually found the real reason to do so, then it shouldn't, otherwise the overall performance would be worse.
As the model gets better, we don't just see a better R^2 (dots line up better), but we also see the slope increase, because the model gets more confident about deviating from the mean.