regression network that will be optimised on a subset of the data
Your reasoning sounds similar to curriculum learning: the idea is to pass training data to the network not randomly, but based on some scoring function $f$. It was proven that a network learns faster and better, even though the process itself is not well understood. I think it is a better approach than the one you suggested: I am afraid that, if you pass only poorly predicted data after a while, the network will forget what learned before.
What you can also do is to perform data augmentation to compensate your unbalanced data (if I understood well, higher ranges in your dataset are not well represented compared to a certain range).