title: How to calculate confidence score of a Neural Network prediction
body: <p>I am using a deep neural network model (implemented in <code>keras</code>)to make predictions. Something like this:</p>
  
  <pre><code>def make_model():
   model = Sequential()       
   model.add(Conv2D(20,(5,5), activation = "relu"))
   model.add(MaxPooling2D(pool_size=(2,2)))    
   model.add(Flatten())
   model.add(Dense(20, activation = "relu"))
   model.add(Lambda(lambda x: tf.expand_dims(x, axis=1)))
   model.add(SimpleRNN(50, activation="relu"))
   model.add(Dense(1, activation="sigmoid"))    
   model.compile(loss = "binary_crossentropy", optimizer = adagrad, metrics = ["accuracy"])
  
   return model
  
  model = make_model()
  model.fit(x_train, y_train, validation_data = (x_validation,y_validation), epochs = 25, batch_size = 25, verbose = 1)
  
  ##Prediciton:
  prediction = model.predict_classes(x)
  probabilities = model.predict_proba(x) #I assume these are the probabilities of class being predictied
  </code></pre>
  
  <p>My problem is a classification(binary) problem. I wish to calculate the confidence score of each of these <code>prediction</code> i.e. I wish to know - Is my model 99% certain it is "0" or is it 58% it is "0".</p>
  
  <p>I have found some views on how to do it, but can't implement them. The approach I wish to follow says: "With classifiers, when you output you can interpret values as the probability of belonging to each specific class. You can use their distribution as a rough measure of how confident you are that an observation belongs to that class."</p>
  
  <p>How should I predict with something like above model so that I get its confidence about each predictions? I would appreciate some practical examples (preferably in Keras).</p>