How do I determine the most appropriate classifier for a certain problem?
This is one of the main skills that separates someone with a deep understanding of, and experience in, machine learning learning, from a neophyte. There are several approaches:


Try several methods, perhaps with automated hyperparameter optimization, and see if there's a big difference in typical model quality. This is pretty common if you don't have a lot of experience, but also something experts may try in a more targeted way.
Visualize the shape of your problem, perhaps by using a dimensionality reduction technique like PCA or tSNE, or maybe an auto-encoder. If you compress the data to 2d, are there clear linear patterns? Maybe try a linear model like logistic regression. Are there several distinct groups? How many lines would you need to draw to separate them? If it's a lot, maybe you're going to need a very non-linear model. If it's just a few, maybe a small multi-layer perceptron can help. If there are spiral bands or circular shapes, maybe and SVM with a non-linear kernel. Knowing how to translate the visualizations into intuition about the kinds of model that can help is an advanced skill. You need to understand what shapes of patterns each kind of model can learn to fit, and how these do or do not translate into a higher dimensional space.
Read the literature. If you're working in computer vision, you should try a CNN. Why? Well, everyone else is using them. They work great on most computer vision problems. They hold most competition records. It'd be silly not to try them. If you're working on spam classification though, CNNs are a bad choice. People use compression classifiers, Bayesian models, and sometimes multi-stage models to build complex features. If you look at the recent literature in your area, you can tell what to use. Reading and understanding this literature well enough to interpret it usually requires more advanced scientific training in ML and/or the application area. 


As a final note, the No Free Lunch theorems for ML tell us that this is always going to be an art (at least, that's one interpretation experts argue for), and not a science, so the more practice you get with it, the better you'll become.