Can setting of different thresholds help in model performance in case of handling class imbalances? In a binary classification problem where there is class imbalance, after applying undersampling/oversampling or SMOTE techinques, is it still convention to use a 0.5 threshold if we are making the two classes completely balanced? Or should we still change the threshold based on what we are trying to optimize?
If changing the threshold improves the performance of the model, it would be better to change it and use an optimal value. Balancing the dataset does not mean that you can not change the discrimination threshold. Also, in some cases when the data is originally balanced, changing the threshold could be very useful and it is a very smart move.