Text classification analysis based on similarity
I shall give you some hints here.

It depends on your data, my advice is that you try several algorithms. 

If you have binary classification you can use both classification (try Logistic regression, SVM, RandomForest) or regression (try XGBoost or any gradient boosting algorithm, there are several in scikit learn), and then seek for a boudary in order to decide wheter a value is 0 or 1. Depending on your data, it might work very well (I've got 0.8 fscore on text classification using RandomForest and performing an important pre-processing before). Do not forget to do an appropriate preprocessing. Concerning pre-processing, in scikit-learn you have CountVectorizer and TfIdfVectorizer, very useful.

Do not use Naive Bayes, not on text and NLP.

Well, the best method for that is using Sequence models (especially LSTM), parsing text as an succession of words or characters. You can use in this case Sequences available in Keras.