Undersampling improvement F1-score Im doing a 2-class classification project for an imbalanced data set.
The imbalance is about 18%/82%. Im noticing a huge improvement in F1-score when I under-sample; from 16% without under-sampling to 49% with under-sampling. Im wondering if this is possible or that I am doing something wrong? Before under-sampling I split the data set in a train- and test set, apply outlier detection and normalisation (normalisation first on train set, then on test using the scales obtained from train set). Does anybody have any idea, if this improvement is too good to be true, what could be going wrong?? Many thanks in advance!
Improvement in the evaluation metric by rebalancing depends on the type of classifier algorithm. Certain algorithms are very sensitive to rate imbalances, thus adjusting the respective levels of support will change the algorithm's performance.

One way to empirically verify that only under-sampling is driving the improvement is to run an experiment - gradually increase the level of under-sampling and plot the result on the F1 score.