Text Classification - Handling text that does not fit into any category
Do a texts sometimes have multiple categories? -> Check multi-label classification

In multi-label classification a text can be either in zero, one, or more than one categories. A simple implementation is a One-VS-Rest Approach. (Think of a classifier for each Category, classifying either True or False, if all these classifiers classify False, the sample isn't in any category.) You can also select to class with the highest confidence when multiple classes were predicted and you only want to have a single one.

Do a text only have zero or one category? -> Add a No Category Class,

this no category class shall be predicted for samples without category. Of course you need to have training samples for this approach.
And of course your approach with a threshold makes sense as well, but it is difficult to tune this threshold to a perfect level, so I would use either a no category class or a multi label approach.
I am working on developing a text classifier and have found some solid options. However the one thing I am struggling with is handling text that does not fit into any predefined categories. This is definitely going to be something we see with real data so I need to understand how to handle it.
I have noticed that when I look at the predict_proba output, (for naive bayes/boosting) the values need to add up to 1. I think this is a problem because if the text meets no category, the algorithms still require that output to equal one, and it will still assign some arbitrary label to that text, even though it really should not.
I have come up with some solutions to take the max probability if within a .90 threshold and assign that category, but I feel this is probably not the best approach.
Does any one have any suggestions on some methods I may be able to try to solve for this?
Thanks