Machine Learning: Balanced training set but highly unbalanced prediction set? How to adjust?
Over- or undersampling should be your second choice. Currently, the best method to deal with class-imbalances is to use the weights argument which sklearn- and keras-classifiers support (see for example the DecisionTreeClassifier).

On a general sidenote, I would recommend focusing on the f1-score,  AUC, and the confusion matrix to evaluate the model's performance, as accuracy might be not as informative in this case.