title: How to interpret almost perfect accuracy and AUC-ROC but zero f1-score, precision and recall
body: <p>I am training ML logistic classifier to classify two classes using python scikit-learn. They are in an extremely imbalanced data (about 14300:1). I'm getting almost 100% accuracy and ROC-AUC, but 0% in precision, recall, and f1 score. I understand that accuracy is usually not useful in very imbalanced data, but why is the ROC-AUC measure is close to perfect as well?</p>
  
  <pre><code>from sklearn.metrics import roc_curve, auc
  
  # Get ROC 
  y_score = classifierUsed2.decision_function(X_test)
  false_positive_rate, true_positive_rate, thresholds = roc_curve(y_test, y_score)
  roc_auc = auc(false_positive_rate, true_positive_rate)
  print 'AUC-'+'=',roc_auc
  
  1= class1
  0= class2
  Class count:
  0    199979
  1        21
  
  Accuracy: 0.99992
  Classification report:
               precision    recall  f1-score   support
  
            0       1.00      1.00      1.00     99993
            1       0.00      0.00      0.00         7
  
  avg / total       1.00      1.00      1.00    100000
  
  Confusion matrix:
  [[99992     1]
   [    7     0]]
  AUC= 0.977116255281
  </code></pre>
  
  <p>The above is using logistic regression, below is using decision tree, the decision matrix looks almost identical, but the AUC is a lot different.</p>
  
  <pre><code>1= class1
  0= class2
  Class count:
  0    199979
  1        21
  Accuracy: 0.99987
  Classification report:
               precision    recall  f1-score   support
  
            0       1.00      1.00      1.00     99989
            1       0.00      0.00      0.00        11
  
  avg / total       1.00      1.00      1.00    100000
  
  Confusion matrix:
  [[99987     2]
   [   11     0]]
  AUC= 0.4999899989
  </code></pre>