Similar SO questions:

* [Python Catboost: Multiclass F1 score custom metric](https://stackoverflow.com/questions/61354410/python-catboost-multiclass-f1-score-custom-metric)

Catboost tutorials

* <https://catboost.ai/docs/concepts/python-usages-examples.html#user-defined-loss-function>

Question
========

In this question, I have a binary classification problem. After modelling we get the test model predictions `y_pred` and we already have true test labels `y_true`.

I would like to get the custom evaluation metric defined by following equation:

```
profit = 400 * truePositive - 200*fasleNegative - 100*falsePositive

```

Also, since higher profit is better I would like to maximize the function instead of minimize it.

How to get this eval\_metric in catboost?

Using sklearn
=============

```py
def get_profit(y_true, y_pred):
    tn, fp, fn, tp = sklearn.metrics.confusion_matrix(y_true,y_pred).ravel()
    loss = 400*tp - 200*fn - 100*fp
    return loss

scoring = sklearn.metrics.make_scorer(get_profit, greater_is_better=True)

```

Using catboost
==============

```py
class ProfitMetric(object):
    def get_final_error(self, error, weight):
        return error / (weight + 1e-38)

    def is_max_optimal(self):
        return True

    def evaluate(self, approxes, target, weight):
        assert len(approxes) == 1
        assert len(target) == len(approxes[0])

        approx = approxes[0]

        error_sum = 0.0
        weight_sum = 0.0

        ** I don't know here**

        return error_sum, weight_sum

```

Question
========

How to complete the custom eval metric in catboost?

UPDATE
======

My update so far

```py
import numpy as np
import pandas as pd
import seaborn as sns
import sklearn

from catboost import CatBoostClassifier
from sklearn.model_selection import train_test_split

def get_profit(y_true, y_pred):
    tn, fp, fn, tp = sklearn.metrics.confusion_matrix(y_true,y_pred).ravel()
    profit = 400*tp - 200*fn - 100*fp
    return profit

class ProfitMetric:
    def is_max_optimal(self):
        return True # greater is better

    def evaluate(self, approxes, target, weight):
        assert len(approxes) == 1
        assert len(target) == len(approxes[0])

        approx = approxes[0]

        y_pred = np.rint(approx)
        y_true = np.array(target).astype(int)

        output_weight = 1 # weight is not used

        score = get_profit(y_true, y_pred)

        return score, output_weight

    def get_final_error(self, error, weight):
        return error

df = sns.load_dataset('titanic')
X = df[['survived','pclass','age','sibsp','fare']]
y = X.pop('survived')

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=100)

model = CatBoostClassifier(metric_period=50,
  n_estimators=200,
  eval_metric=ProfitMetric()
)

model.fit(X, y, eval_set=(X_test, y_test)) # this fails

```