What is a better solution for text classification than use of perplexity
You can evaluate pre-built language models. Advantage is :


These models are usually trained on very large text corpora (Such a a snapshot of  text from billions of Web pages). 
It requires very little development effort for testing models for a given classification problem. 


Examples :

https://spacy.io/usage/examples

https://uber.github.io/ludwig/examples/