How to maximize recall? I'm a little bit new to machine learning.
I am using a neural network to classify images. There are two possible classes. I am using a Sigmoid activation at the last layer so the scores of images are between 0 to 1.
I expected the scores to be sometimes close to 0.5 when the neural net is not sure about the class of the image, but all scores are either 1.0000000e+00 (due to rounding I guess) or very close to zero (for exemple 2.68440009e-15). In general, is that a good or bad thing ? How can this behaviour be avoided?
In my use case I wanted to optimize for recall by setting a lower threshold but this has no impact because of what I described above.
More generally, how can I minimize the number of false negatives when in training the neural net only cares about my not ad-hoc loss ? I am ok with decreasing accuracy a little bit to increase recall.
To answer the last question, suppose that you have a binary classification problem. It is customary to label the class as positive if the output of the Sigmoid is more than 0.5 and negative if it's less than 0.5. For increasing recall rate you can change this threshold to a value less than 0.5, e.g. 0.2. For tasks which you may want a better precision you can increase the threshold to bigger value than 0.5. 

About the first part of your question, it highly depends on your data and its feature space. There are problems which the data is linearly separable in higher dimensions which means you can easily employ just a single neuron for classifying the data by a single hyper-plane. If it has happened that you have such a good accuracy you can not say anything unless you try to find the value of cross validation error. By interpreting the difference between the value of training data, and cross-validation or maybe test data, you can figure out whether your classifier performs well or not.