Should the cost function be zero using TensorFlow's sigmoid_cross_entropy_with_logits?
For a binary classification (1 or 0), you should use softmax cross-entropy with logits since the output classes are mutually exclusive / discrete.

The function you used is for cases with multiple outputs (multilabel) all of which can be ones (their documentation for example talks of the case where image can contain both elephant and a dog). The calculation is different than mutually exclusive case.