What does non-linearity mean in Neural Networks? Why is it necessary?
A neural network, in its simplest form, consists of layers of neurons.
The first layer of neurons are the input, and the last layer forms the output.
A neuron $x$ on layer $i+1$ is connected to several neurons $y_1,\ldots,y_d$ on layer $i$, and it computes the following function:
$$
x = f(\alpha_1 y_1 + \cdots + \alpha_d y_d + \beta),
$$
where $f$ is a fixed function (nowadays often ReLU), and $\alpha_1,\ldots,\alpha_d,\beta$ are parameters which are learned in the training period. (The output layer often uses a different $f$.)
If $f$ is the identity function $f(z) = z$, then each neuron in the output is just an affine function of the input. This means that no matter how many layers the neural network has, all it can do is compute linear functions.
In order to allow the network to compute more complicated functions, we introduce nonlinearity, in the form of a non-linear function $f$ (that is, a function which is not of the form $f(z) = az + b$), such as ReLU, which is $f(x) = \max(x,0)$. This makes the neural network more expressive.