Algorithm selection rationale (Random Forest vs Logistic Regression vs SVM)
I suppose I will suggest as a starting point and expand on what you suggested by just adding the following


Knowing the type of data you are working with and it's characteristics, (categorical, supervised/unsupervised, data size etc.). 
Knowing what accuracy requirements you need, timeframe and computational power you have at your disposal vs accuracy and really answering "why, am I trying to solve this problem?"


After answering these questions you can at least narrow down slightly what you may use (and eliminate those you clearly don't believe fit).  After that, I suppose it's trial and error, experience and comparing to others who dealt with similar datasets and problems.

I have this crude flow chart I found in my favourites from the scikitlearn website. Not sure where I found it to be honest.  Take it for what you will, hopefully it helps somewhat:

https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html