Troubleshooting Neural Network Implementation
Are your inputs scaled? Not doing so can cause the weights to immediately blow up. 

It's common practice to preprocess your input to be between -1 and 1 or at least in that range. Otherwise you are running the risk of your gradients exploding or vanishing (covered here).