Hand computing feed forward and back propagation of neural network
One important point I missed in first review: error is a summatory, its derivative is also a summatory.

About offsets "b": usually they are different in each cell (if not fixed to some value, as 0). Thus, replace them by b1 and b2. Moreover, they should be optimized in the same way that the weights.