Neural Network Backpropagation problems I'm working on an implementation of a neural network so I can really grasp how these magic boxes work. However the neural network I've written code for doesn't work and I think it's due to my implementation of backpropagation

[pseudo code]

'last layer only
for each node (i) in last layer
error(i) = outputError(i)
end for

'all hidden/inner layers
for each layer (i) in layers (not including layer 0 or last)
 for each neuron (j) in layer i
  for each neuron (k) in layer i+1
   error(i)(j) +=error(i+1)(k) * weight(i)(j,k)
  end for

  errors(i)(j) = errors(i)(j) * activation(i)(j) * (1-activation(i)(j))

  for each neuron in layer i+1
   deltaWeight(i)(j,k) = -error(i)(k)*activation(i)(k)*learningRate/NumExamples
   deltaBias(i)(k) = -error(i)(k) * bias(i)(k)*learningRate/NumExample
  end for
 end for
end for


Then I update all the weights with this:

weight(i)(j,k) = weight(i)(j,k) + deltaWeight(i)(j,k)


Now the problem I have is that the output doesn't seem to get any better. It certainly changes but it doesn't seem to minimise the cost function at all, does anyone know why?
Some reasons the network may not be getting better: 


Learning rate is too high (jumps back and forth forever)
Learning rate is too low (when divided by training set size, goes to zero)
You're replacing the error signal with the weight gradient signal.  The reuse of error (based on the indentation) seems to be right on the same line as the inner loop.  I'm not sure if that's correct.  First, backprop your errors, THEN calculate the gradient based on the activation of the hidden neurons and delta sigmoid(error).