Classifier performance evaluation
Useful metrics in such scenario are: 


F1 Score (and precision / recall)  
ROC Curves  (Metric is : Area Under the ROC Curve (AUC))


Few articles on how to choose metrics for a specific project are: 


Evaluation Metrics, ROC-Curves and imbalanced datasets by David S. Batista,
What metrics should be used for evaluating a model on an imbalanced data set? by Shir Meir Lador,
Choosing the Right Metric for Evaluating Machine Learning Models — Part 2 by Alvira Swalin.