Looking for other opinions on approach to classification problem I'm looking to implement an "opt-out" filter for my company. The input is short, text-message style messages. A few examples of opt-out messages are:


"remove me from your list"
"remove from list"
"please unsubscribe from list"
etc.


All other messages are "good", and should not be removed.

My thoughts on approaches:

I was thinking of using a Bayesian classifier here, but not really knowing the solution space (or having much of a background in ML), want to be sure I'm not wasting time on a sub-optimal solution.

I'm fine with not having the most cutting-edge solution, but want to be sure I'm not missing an approach that might be equally as straightforward but more effective.
You should use text classification techniques. The most basic one is multinomial naive Bayes classifier with tf-idf features. for this method, take a look at this:
https://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_data.html

If you don’t get enough accuracy (or maybe precision, recall or f-score), you could test more complex techniques e.g. using deep LSTM networks with word embedding. For this method, take a look at this:
https://machinelearningmastery.com/use-word-embedding-layers-deep-learning-keras/