Oversampling only balances the training set, what about the testing set?
Use a stratified split (as pointed out by louic in the comments). It will distribute your classes evenly across all folds. It can be done using sklearn's StratifiedKFold.