Overall AUC higher than all "stratified" AUCs For one of my binary classification models, I have observed this (Simpson's Rule-esque) paradox.  The AUC on the test set as a whole is 0.8.
Gender is one of the model's features.  So I decided to produce a &quot;bias&quot; report, for which I calculated AUCs for each of the Male and Female subgroups.  But I noticed that each of these AUCs is around 0.7.  How is this possible, given that the overall test AUC is 0.8?  (In my dataset, every data point belongs to either the Male or the Female subgroup.)  I don't expect the overall AUC to simply be a (weighted) linear combination of AUCs for the individual strata.
I'm hoping to get both a technical/mathematical answer and a high-level explanation.  Please let me know if any further information is needed (if you think I should plot the overall, Male, and Female ROC curves, for instance).  Thank you!
AUC can be defined as $P(X_1 &gt; X_0)$ where $X_1$ is the score of a randomly chosen positive instance and $X_0$ is the score of a randomly chosen negative instance.
Like in Simpson's &quot;paradox&quot;, what you see could happen because the group has a relatively large effect on your target. For example, one group could be mostly positive, another could be mostly negative. The classifier might not work at all within each of the groups, but if it is merely able to predict the group we are in (or as in your case it is simply one of the inputs), it can have a high AUC.