how to choose classifer
It's usually not that clear cut; there's typically not one universally best approach.

Having said that, there are some prototyped ensemble approaches that are supposed to always be better than their underlying component algorithms, notably Erin LeDell's binary ensemble classifier for H2O. However, even in those cases you still need to optimize the first stage algorithms for the ensemble to be universally better.

Thus if you're willing to spend a lot of extra time, let's say 2 weeks for an ensemble instead of the 1 week it might take you for your single-stage algorithm, then it's possible (especially for binary classification) to find an ensemble that will definitely be better than your single-stage classifier.

However this is rarely the case and the way you framed the question implies there's a choice between 


building 1 really good single-stage model, selected from many candidate models (and by the way be sure to avoid overfitting while making those selections)  and
throwing an ensemble at the problem without completing #1 above for each component of the ensemble (or completing #1 but not also optimizing the 2nd stage of the ensemble)


If that's the decision then -- while there's no 1 universally right answer -- I'd say that in the vast majority of the cases it's better to stick with #1.