Neural network is getting partially trained
I can see a problem with your error function (blue). It is treating under-predictions and over-predictions the same. The error gradient should ideally direct you towards zero error.



For your error function:

$$
E = 2 \cdot (y_{expected} - y_{predicted})
$$

The derivative w.r.t prediction is:

$$
\frac{dE}{d\;y_{predicted}} = -2
$$

Note here that the error is the same whether you under-predict or over predict (i.e. the sign of error doesn't matter in the gradient). So the updates to the weights will be in the same direction (see arrows on blue line).

But if you use a quadratic error function (orange line), over- and under- prediction errors are treated differently (i.e. the gradient depends on the sign of the $expected - predicted$ difference).

$$
E = (y_{expected} - y_{predicted})^2 \\
\frac{dE}{d\;y_{predicted}} = 2 \cdot (y_{expected} - y_{predicted}) \cdot -1
$$