Why is the input to an activation function a linear combination of the input features? I'm new to deep learning, and there is something about the calculations I do not understand:
For a standard neural network, why is it that only the activation function is not linear, but the input to the activation function is a linear combination of each of the $x_i$'s? For example, with the sigmoid function, it would look like:
$$ \frac{1}{1+ e^{-(w_0x_0 + w_1x_1 + b)}} $$
where $w_i$ are the weights and $x_i$ represents the input to that layer.

For example, why is it that we don't have something like this:
$$ \frac{1}{1+ e^{-(w_0x_0^2 + w_1\sqrt{x_1} + b)}} $$

Is it because it would be redundant if we had enough layers? Or is it because a priori, you wouldn't know what the best function is?
The main reason is that a linear combination of the input followed by a non-linearity stacked on top of eachother is a universal function approximator. Which means that no matter how complicated the true underlying function is, a neural network can approximate it to an arbitrarily small error.

There's also the efficiency factor since a linear combination of $n$ inputs each having $m$ dimensions can be represented using a single matrix multiplication $h=X \times W$ where $X$ is an $n \times m$ matrix (where each row is an example and each column is a feature of that example) and $W$ is an $ m \times d $ weight matrix. And computers are VERY efficient at doing matrix multiplications. Thus, the more you build your model to use matrix multiplications the better.