How do I know if my backpropagation is implemented correctly?
For future reference, I will merely point you to a technique you can implement to test the correctness or lack thereof, of your backpropagation implementation.

Ps: don't feel too bad for having gotten it slightly wrong, "backpropagation is notoriously difficult to implement" - source :). 

In fact, there is a technique called "Gradient checking" meant specifically for this purpose (for more information, see Andrew Ng's lecture video on gradient checking and, this notes). I would argue that even gradient checking is a little tricky to implement.  

How does Gradient checking work: 


Backpropagation computes the gradients (∂J/∂θ)(∂J/∂θ), where θ denotes the parameters of the model. J is computed using forward propagation and your loss function.
But because forward propagation is fairly straightforwards to implement, most people are usually confident that you got its implementation correct. So the trick is to use the value of J to verify your code for computing (∂J/∂θ)(∂J/∂θ). 
We know that by definition, the gradient or derivative is given by:

(∂J/∂θ) = lim ε→0 J(θ + ε) − J(θ − ε) / 2ε 



now, since we trust our calculation of (∂J/∂θ), we can easily compute the value of J(θ+ε)J(θ+ε) and J(θ−ε)J(θ−ε).

For more information, see here and, over here).