What is wrong with my logistic regression implementation?
What is the ratio between 0's and the other numbers? If the ratio is too low, let's say 2%, if you classify everything as not 0 you will have high accuracy.

EDIT AFTER COMMENTS

Imagine there are 98 non zeroes and 2 zeroes, you don't need any classifier to have a 98% of accuracy... 

Accuracy is not the best metric for unbalanced data-sets, let's try precision and recall and take a look to the confusion matrix