Why do we need activation function (like ReLU) after an affine layer?
This affine operation $y = W^{\top} x + b$ has already add nonlinearity to the system given that $b \neq 0$.


This is not considered a non-linearity in the context of data science. Different disciplines define linearity sometimes in subtly different ways. Critically, the $+b$ performs identically in terms of fitting to data, as extending $x$ with a new dimension, always $1$, and moving the values of $b$ into the weights $W$. This simpler multiplication is clearly linear.

Also importantly, the affine transformations form a group such that any two affine transformations combined are just another affine transformation with different parameters. Without a non-linearity in a hidden layer, a 2-layer neural network would be the same as a single layer one, and not able to learn whole classes on non-linear relations.

No matter how many affine transformations you apply to inputs for instance, you will not be able to approximate the XOR function, or any significant portion of $y=\text{sin}(x)$