Finding an optimum back propagation algorithm
When you are training a neural network, you use an algorithm called back propagation. This algorithm uses partial derivatives to determine the optimal values for weights. Partial derivatives are a calculus based method which tell you how far you need to adjust the weights in order to get to an optimum value. However, when you have neural networks with many different weights you need to be able to decide which ones to update in order to achieve a more optimal network. This is where stochastic gradient descent comes in. This algorithm is able to take many different gradients and traverse through many dimensional space into a lower error rate. Also, from your diagram of your neural network, I do not see any hidden neurons, which are neurons between the input and output layers which do most of the actual work. While you might be able to train your neural network optimally without hidden neurons, it might improve your accuracy depending upon your dataset. Finally, implementing a back propagation algorithm is a lot of work which is why many people use pre built libraries. Unless you are doing research, I recommend you do the same. I think for a problem like this, pybrain would be perfect, and if you want to use deep neural networks, tensorflow would be good for that.