Document classification - optimal classifier & embedding Planning a classifier to recognize specific documents based on text. Categories are mutually exclusive and include: job application resumes, paychecks, quarterly financial reports etc, and data is text extracted from doc, pdf, xls etc.

At the feature engineering stage, with questions in mind:


One Multiclass classifier or a number of One vs. Rest (one per category)?
Word embedding (Word2Vec/GloVe) - one embedding for all categories or a separate embedding for each category?


My tendency is towards a number of one vs. rest classifiers (to allow for dynamic use of just some of the categories), and one embedding for all categories (to save prediction time).

Project architecture: initially basic LogReg, but will soon develop to CNN / CNN+RNN.

Project scale:


Training: a few hundreds per category
Prediction: millions of files


Please feel free to share any insights or references, thank you
I would suggest to have a look at fasttext, it will learn word embeddings for you from your corpus (advantage over word2vec embedding is that it learns representations based on character ngrams, so if a word is there, but its plural is not, it will still have a similar representation) and then in the supervised learning mode it uses something very simple like regression for classification, provided you have examples of labelled data.
I tried it with trivial 20newsgroups and it works well for distinct categories. Maybe not a production-friendly solution (it's in c++, but can be stretched to python, albeit not without pain), but can give an idea.
Given the nature of the algorithm, it's quick, compared to nets.