Trying to understand why nonlinearity is important for neural networks by analogy
Your analogy is correct, except it is not really an &quot;analogy&quot;. Sin is an activation function - in past works (before modern deep learning boom) it was rather standard to see it listed as a possible activator.
So your expression $\sigma(x) = A\sin ax + B \sin bx + D \sin ex$ is of a neural network with one 3-neuron layer and a single output linear neuron:
$$\sigma(x) = \sum_i V_i \sin\left(\sum_jW_{ij} x_j + b_i\right)+\beta$$
With all biases being zero $b_i=\beta=0$, output weights are $V_i = \left(A,B,C\right)$ and the inner weight matrix is diagonal:
$$W_{ij} = \begin{pmatrix}a&amp;0&amp;0\\0&amp;b&amp;0\\0&amp;0&amp;e\end{pmatrix}$$