Over-sampling: is my model over-fitting?
In order to get accurate results, you should not oversample the test set! Otherwise you are simply evaluating on synthetic samples that you yourself have created. The support on your classification report should mirror the imbalance in your dataset.
From what I understand you have 3500 samples, then you did some oversampling (probably brought them to around 6000) and then took 1757 from these for testing. This evaluation scheme is wrong. Take a look at the illustration below to see a more correct scheme.
      |--- train --&gt; oversample train set --&gt; train model---|
set --|                                                     |--&gt; evaluation on test set
      |--- test --------------------------------------------|