ROC curve shows strange results for imbalanced dataset I have a classifier with a heavily imbalanced dataset (1000 of each negative label for each positive.)

I'm running a GradientBoostingClassifier with moderate success (AUC .75) but the curve has this strange look:



Any good ideas on what would cause the curve to have this behaviour?
Davis and Goadrich have explained the relationship between ROC and PR Curves in their paper. It is always recommended to use PR curve over the ROC curve in the presence of highly imbalanced data. 

Back to the behavior of your ROC curve, It seems that you don't have more threshold points! I would also agree with Dan and do K-fold CV. 


  Davis, J. and Goadrich, M., 2006, June. The relationship between
  Precision-Recall and ROC curves. In Proceedings of the 23rd
  international conference on Machine learning (pp. 233-240). ACM.