How does a single hidden layer affect output?
When we add a single layer with a non-linear activation function, right after the application of the activation function, a new basis function is found (for that neuron), which is some combination of the weights and biases, which acts as a new way to view or analyze the feature sets. 

With an increasingly deeper network, we keep finding representations, which are new basis vectors of the combination of the previous layer's weights and biases, that is, higher-level representations. 

If they're error-free, they'll give better performances, but if small errors creep in earlier basis vectors, the error increases through depth.

A nice analogy is the Taylor series, where $1$, $x$, $x^2$, and so on, are the basis vectors for estimating the function in 1D.