AUC higher than accuracy in multi-class problem I stumbled upon a 3-class classification problem where all compared classifiers yield a higher AUC than accuracy (usually around 10% higher). This happens both when the dataset is balanced or slightly imbalanced.
Now, after looking at this answer: Why is AUC higher for a classifier that is less accurate than for one that is more accurate? I understand that, for binary classification, this might happen because the accuracy is typically computed at a threshold of 0.5 whereas AUC is based on all threshold values.
But what happens with multi-class classification? Specifically, those scenarios where accuracy is defined as the frequency with which the predicted labels match the true labels (tf.keras.metrics.CategoricalAccuracy) and AUC is defined as the weighted average of the AUC for each class vs the rest (One-vs-rest) (sklearn.roc_auc_score). Why might AUC be higher there?
In other words, I'm trying to understand what this result means. Does it mean that my classifiers can discern well when each class is measured against the others (AUC), but not as well when the prediction probabilities are an output of the softmax function, and therefore spread out for the three classes?
I think the question can se splitted to three parts. The first part is about comparing accuracy and AUC of the same model, the second is about comparing models and the third is about multi-class problems.
First part - I think that accuracy and AUC are not comparable metrics, I don't think that there is special meaning to the case where one is higher than the other.
Second part - ROC plot and AUC are good for understanding model and choosing model types, but in the end of the day you usually want to evaluate your final model with specific threshold so accuracy, precision and recall are the relevant metrics.
Third part - I think the answers to the previous parts above are valid both for binary classification and multi-class classification. Why do you think it have different meaning here?