How to improve an existing (trained) classifier?
In order to improve your classifier, you have few options.


Ensembling - make a group of classifier and let them predict together. Stacking, blending, bagging, boosting. Choice is yours.
Hyper parameter tunning - you did not mention your tool, but i suppose that in every solid one is option to search in parameter space to find the best combination
Sampling - you can try undersample or create new samples (SMOTE) to give your classifier more data of class which you'd like to predict
Feature engineering - get rig of noisy features and work only with those which have an effect on your prediction, consider PCA too
Scaling - normalize your data can improve the performance, some classifiers require this
Data quality - missing values, coding categorical variables or suspicious values can influence performance