How do neural networks manage to do regression?
I came across the article A visual proof that neural nets can compute any function that explains how a neural network can approximate any function, which pretty much answers my question.

I suppose connections between neurons do not generally arrange themselves in the way described, though, so it would still be interesting to get insight into whether there are patterns of how the inter-neuron connections arrange themselves in neural networks trained to solve actual problems.