How to use downsampling and configure class weight parameter when using XGboost for imbalanced classification?
The class weights are used when computing the loss function to prevent the model from giving importance to the major class. If one class dominates the dataset, then the model will be biased to learn that class better because the loss is mainly determined by the model's performance on that dominant class.
Let's consider an extreme case where the dataset contains 99 percent positive samples. If a model just predicts 1 for every sample, it will have 99 percent accuracy. The idea behind class weights is that you want every sample to contribute to the loss equally. Therefore, you should compute this ratio based on your training set because the loss is computed on your training set. Your model has no idea about the samples that you dropped.
If you make a correct prediction, the loss is 0, and otherwise not. Coming to your case, to make sure that every sample contributes to the loss equally, a false prediction for the minority class should be penalized 4 times more than a false prediction for the majority class. So that, the model can not ignore a certain class or have a bias towards the majority class.
It is generally a good idea to set the class weight anti-proportional to the number of samples you have for that particular class. So, in your case, that would be 4. However, in practice, you should probably try out few different values to find the best weights.
Another important aspect is the ratio of these samples in the wild. You said that you made down-sampling, if the ratio of classes differs in the wild compared to your training dataset, then you might observe worse scores when you deploy your model or when you are testing it on unseen samples. That is why you should ideally also split your validation and test sets with realistic ratios using your domain knowledge
I am working on binary classification problem on a  dataset with extreme class imbalance. To help the model learn the signals of the minority class, I downsampled the majority class such that the training set has  20% of minority class and 80% majority class.
Now there is one other parameter "scale_pos_weight" . I am not sure how to set this parameter after downsampling.
Should i set this based on the actual class ratios or should i use the class ratios after downsampling?