How to improve Naive Bayes?
Without looking at the actual data, all we can really do is second guess and suggest best practices. Here are some pointers you can pursue - 


Gather more data - If it can be done, nothing like it. 
Improve data quality - The algorithm will always be as good as the data. Try extensive cleaning methods like - 
Lowercase (basic) so all your data is standardized
Stemming / Lemmatization - These techniques reduces a word to its root form
Try parts of speech tagging (POS) and retain important parts of speech like nouns and look at their overall importance
Stopword removal - Just like you tried doing word frequencies, there are popular word sets like the word the that don't help much. They are filtered out
Correcting Spelling mistakes - If you are expecting there will be spelling mistakes, might be a good idea to correct them
Converting the unstructured data - I see that you have used bigrams. Try using unigrams and trigrams as well, or in combinations, run your algorithm and see which one works better. Try CountVectorizer, TfidfVectorizer and other techniques like word embeddings as well
Algorithms - Finally focus on the algorithm itself. For Naive Bayes, focus on MultinomialNB. Try RandomForestClassifierand other ensemble family algorithms. Try Deep Learning techniques with keras. Fine tune hyperparameters based on the validation results


Of course, there are other best practices like splitting your data into train, test and cross validation sets. Evaluating the right metrics such as accuracy, precision, recall and the confusion matrix.

Hope this helps!