How do I compare traditional classifiers performance with my proposed method?
What about doing cross validation on your training set? Once you have the different train/test splits I would start by printing the accuracy (number of correct predictions / total predictions) and the confussion matrix for each method.

If you are using python sklearn.metrics offers a wide variety of useful functions.