Training a classifier with text and numerical features - what is the state of the art? I'm trying to build a binary classifier where the features are mostly numerical (about 20) and there are a couple of unstructured short text fields as well. What is currently considered the state of the art for combining these types of features?

I've tried building a separate classifier (logistic regression, TFIDF) using the text data alone, and then including that output score of that classifier as an additional when training using the rest of the numerical features (random forest, taking care to train each classifier on different folds of the data to prevent signal leakage). It works alright, but I think a better performance is possible.

A variant is to simply train two separate classifiers, one using text and the other using numerican features and then stacking those two. And finally, another idea is to use neural networks, have two input networks, one CNN/LSTM for the text, and another dense for the numerical features, and then combining them and having a single output.

Are there other approaches I haven't thought of that are worth a try?
Regarding the first question, to the best of my knowledge, there is no SOTA approach. It depends on the task.

For the second one, did you try to combine the numerical values with the TFIDF vectors, and then feed the final vector(s) to the classifier? Probably not. I usually try this way. But, if your vocabulary is too large, try to reduce it by considering top N-words, because combining small numerical vector with a huge TFIDF vector may not allow the numerical values to affect the results.