How to prevent a neural network from outputing too clustered probabilities? I have a CNN outputting probabilities using a logistic output. The performances are good on the test set. Yet the probabilities it outputs are very clustered, they are either 0 or 1! 



I would like some more granularity in the probabilities outputed by the network. Some features are more obvious than other and the network should be able to learn this. Any idea on how I could solve this problem?
That's usually the most ideal case that you get in classification, but if you really want to, you can add a penalty on the predictions to force them to be away from zero or one, e.g. loss'(x) = loss(x) - param * prob_hat(x) * (1 - prob_hat(x)), or some different penalty.