What is the best way to compare various classification models?
Since the problem you are trying to tacle is image classification, then classification accuracy is the appropriate measure of comparison. Also you could consider the Precision, Recall and F1 metrics (for multi-class problems). See below there is description of an extension about these multi-class metrics.

A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems  

Another appropriate metric is AUC/ROC, which has to be extended cosnidering the multi-class case. See the link below 

Receiver Operating Characteristic (ROC)