title: Custom TensorFlow metric: true positive rate at given false positive rate
body: <p>I have a binary classification problem with categories background (bg) = 0, signal (sig) = 1, for which I am training NNs. For monitoring purposes, I am trying to implement a custom metric in Keras with TensorFlow backend that does the following:</p>
  
  <p>1) Calculate the threshold on my NN output which would result in a false positive rate (classifying bg as signal) of X (in this case X = 0.02, but it could be anything).</p>
  
  <p>2) Calculate the true positive rate at this threshold.</p>
  
  <p>Given numpy arrays y_true, y_pred, I would write a function like:</p>
  
  <pre><code>def eff_at_2percent_metric(y_true, y_pred):
      #Find list of bg events
      bg_list = np.argwhere(y_true &lt; 0.5)
      #Order by the NN output
      ordered_bg_predictions = np.flip(np.sort(y_pred[bg_list]),axis=0)
      #Find the threshold with 2% false positive rate
      threshold = ordered_bg_predictions[0.02*round(len(ordered_bg_list))]
  
      #Find list of signal events
      sig_list = np.argwhere(y_true &gt; 0.5)
      #Order these by NN output
      ordered_sig_predictions = np.sort(y_pred[sig_list])
      #Find true positive rate with this threshold
      sig_eff = 1 - np.searchsorted(ordered_sig_predictions,threshold)/len(ordered_sig_predictions)
  
      return sig_eff
  </code></pre>
  
  <p>Of course, this does not work because to implement a custom metric, y_true and y_pred are supposed to be TensorFlow tensors rather than numpy arrays. Is there any way I can make this work correctly?</p>