What annotators are used in Cohen Kappa for classification problems?
What 2 annotators or observers does Cohen Kappa use in classification problems when you compute the score in scikit learn?


The two annotators are exactly the two parameters you fed the method with: cohen_kappa_score(y_test, predictions), i.e. the truth/labels and your predictions. And these are then measured in terms of their "inter-rater" agreement.