title: Custom macro for recall in keras
body: <p>I am trying to create a custom macro for <code>recall = (recall of class1 + recall of class2)/2</code>. I came up with the following code but I am not sure how to calculate the true positive of class 0.</p>
  
  <pre class="lang-py prettyprint-override"><code>def unweightedRecall():
      def recall(y_true, y_pred):
          # recall of class 1
          true_positives1 = K.sum(K.round(K.clip(y_pred * y_true, 0, 1)))
          possible_positives1 = K.sum(K.round(K.clip(y_true, 0, 1)))
          recall1 = true_positives1 / (possible_positives1 + K.epsilon())
  
          # --- get true positive of class 0 in true_positives0 here ---
          # Also, is there a cleaner way to get possible_positives0
          possible_positives0 = K.int_shape(y_true)[0] - possible_positives1
          recall0 = true_positives0 / (possible_positives0 + K.epsilon())
          return (recall0 + recall1)/2
      return recall
  </code></pre>
  
  <p>It seems I will have to use <code>Keras.backend.equal(x, y)</code>, but how do i create a tensor with shape <code>K.int_shape(y_true)[0]</code> and all values, say x?</p>
  
  <hr>
  
  <p>Edit 1</p>
  
  <p>Based on Marcin's comments, I wanted to create a custom metric based on callback in keras. While <a href="https://github.com/keras-team/keras/issues/5794" rel="nofollow noreferrer">browsing issues in Keras</a>, I came across the following code for f1 metric:</p>
  
  <pre class="lang-py prettyprint-override"><code>class Metrics(keras.callbacks.Callback):
      def on_epoch_end(self, batch, logs={}):
          predict = np.asarray(self.model.predict(self.validation_data[0]))
          targ = self.validation_data[1]
          self.f1s=f1(targ, predict)
          return
  metrics = Metrics()
  model.fit(X_train, y_train, epochs=epochs, batch_size=batch_size, validation_data=[X_test,y_test], 
         verbose=1, callbacks=[metrics])
  </code></pre>
  
  <p>But how is the callback returning the accuracy? I wanted to implement <code>unweighted recall = (recall class1 + recall class2)/2</code>. I can think of the following code but would appreciate any help to complete it</p>
  
  <pre class="lang-py prettyprint-override"><code>from sklearn.metrics import recall_score
  class Metrics(keras.callbacks.Callback):
      def on_epoch_end(self, batch, logs={}):
          predict = np.asarray(self.model.predict(self.validation_data[0]))
          targ = self.validation_data[1]
          # --- what to store the result in?? ---
          self.XXXX=recall_score(targ, predict, average='macro')
          # we really dont need to return anything ??
          return
  metrics = Metrics()
  model.fit(X_train, y_train, epochs=epochs, batch_size=batch_size, validation_data=[X_test,y_test], 
         verbose=1, callbacks=[metrics])
  </code></pre>
  
  <hr>
  
  <p>Edit 2: model:</p>
  
  <pre class="lang-py prettyprint-override"><code>def createModelHelper(numNeurons=40, optimizer='adam'):
      inputLayer = Input(shape=(data.shape[1],))
      denseLayer1 = Dense(numNeurons)(inputLayer)
      outputLayer = Dense(1, activation='sigmoid')(denseLayer1)
      model = Model(input=inputLayer, output=outputLayer)
      model.compile(loss=unweightedRecall, optimizer=optimizer)
      return model
  </code></pre>