How to maximize recall score for specific label in multiclass classification? Thought I had solved the problem but I'm having inconsistent issues with it so reaching out here.
I have a multilabel classification problem with four labels ['--','-','+','++'] and with a basic random forest model, I have significant performance issues with one label '-', while the other three labels are performing pretty decently.
model = RandomForestClassifier(random_state=42)
model_name = 'RFC_base'

grid_params={}

pipe = GridSearchCV(make_pipeline(model),scoring='recall_weighted',cv=5,param_grid=grid_params,n_jobs=-1)

          recall avg    recall (++) recall (+)  recall (-)  recall (--) 
RFC_base    0.848485    0.840909    0.871795    0.771429    0.893617

I'm well aware that GridSearchCV is over engineering, but I like to keep code consistent between tinkers while optimising.
I wanted to improve the recall score for '-', so created a custom scoring function that I thought would be maximising the recall score for '-'.
recall_neg_scorer = make_scorer(recall_score,average=None,labels=['-'],greater_is_better=True)

I've been tinkering with this to optimise it, hence trying to define the 'greater_is_better' param. So I do an actual GridSearchCV now changing some of the params.
model = RandomForestClassifier(random_state=42)
scaler = StandardScaler()

grid_params = {
    'randomforestclassifier__n_estimators': [81], #np.arange(start=60,stop=90,step=1),
    'randomforestclassifier__max_depth': np.arange(1,10,step=1),
    'randomforestclassifier__criterion': ['gini'], #,'entropy'],
    'randomforestclassifier__max_features': [7], #np.arange(1,X.shape[1]+1,step=1),
    'standardscaler': [StandardScaler(),MinMaxScaler(),None]
}

model_name = 'RFC_gscv_recall_neg'

pipe = GridSearchCV(make_pipeline(scaler,model),scoring=recall_neg_scorer,param_grid=grid_params,cv=5,n_jobs=-1)

What I'm finding is my scoring function doesn't actually optimise '-' when 'greater_is_better = True'
For instance, when GSCV runs on 'max_depth' it chooses 9 which gives :
          recall avg    recall (++) recall (+)  recall (-)  recall (--) 
RFC_gscv_9  0.836364    0.818182    0.871795    0.742857    0.893617

Which is performing worse on '-' than even the base case. When I force max_depth = 8 it gives :
          recall avg    recall (++) recall (+)  recall (-)  recall (--) 
RFC_force_8 0.842424    0.818182    0.871795    0.8        0.87234  

When I choose 'greater_is_better = False' it actively tries to minimise the score. Is there something I'm screwing up here or is there a known issue that I'm missing out?
Also I'm a bit new to stack so let me know if there's something I'm missing.
I would highly suggest doing some deeper Analysis on why the (&quot;-&quot;) class is beeing predicted. Meaning what do you predict for class (&quot;-&quot;) when no the correct class. Analysing this you could see that you are ordering class (&quot;-&quot;) as class &quot;++&quot;. Than you can look at the Input data and feature engineer Features that discriminate between These two class. Forcing you model to do this indirectly is a fair try, but model cant learn whats not to be found in the data. With feature Engineering you can enhance the data so that discrimination is better.