Catboost multiclassification evaluation metric: Kappa & WKappa I am working on an unbalanced classification problem and i want to use Kappa as my evaluation metric. Considering the classifier accepts weights (which i have given it), should i still be using weighted kappa or just use the standard kappa? I am not entirely sure of the difference to be honest.
model = CatBoostClassifier(
    iterations = 1000,
    learning_rate = 0.1,
    random_seed = 400,
    one_hot_max_size = 15,
    loss_function = 'MultiClass',
    eval_metric = 'WKappa', # weighted kappa
    #ignored_features = ignore_list,
    class_weights= weights,
    od_type = &quot;Iter&quot;,
    od_wait = 50,
    task_type = 'GPU',
    border_count = 124,
    depth = 12
)
I posted the same question on the Catboost github (issues) page and got an answer.
The link can be found here: https://github.com/catboost/catboost/issues/1447
Answer:
Class weights and weights in WKappa are different.
Calculation of WKappa metric consists of two steps:

Confusion matrix calculation - here object and class weights are used.
WKappa calculation using confusion matrix - here values from confusion matrix multiplied by squared difference between classes. (i-j)^2.
In other words, weight of class 0 has the same effect when object of class 0 was classified as class 1 and when classified as class 5. But WKappa weight for misclassification 0-&gt;5 will be 25 times more than for misclassification 0-&gt;1.
So you can use both class weights and WKappa if your classes are ordinal.