How to prefer no choice instead of bad choice with sklearn decision tree I'm using sklearn decision trees to classify documents in two possible types "type1" and "type2".

I've isolated few features that seem pertinent and tried to combine them manually to evaluate the result of a model.
When classifying the documents manually i use the following outcomes:


type 1
type 2
unknown


Then I give the same features to a decision tree. The results are worse in that case because it always tries to classify documents in one of the categories "type1" or "type2" but is unable to classify documents in "unknown"

Is it possible to configure a sklearn decision tree in a way that in case of high uncertainty the document won't be classified instead of selecting a category that is likely to be wrong?
TL;DR
You can train your classifier as usual and threshold on the prediction probability.
Scikit-learn
You won't get that out of the box, but you can use this simple class to do just that!
from sklearn.tree import DecisionTreeClassifier
from numpy import argmax, max

class MyClassifier(DecisionTreeClassifier):

    unknown_class = 'unknown'

    def __init__(self, no_class_threshold=0.75, **kwargs):
        self.no_class_threshold = no_class_threshold
        super().__init__(**kwargs)

    def predict(self, X):
        preds = self.predict_proba(X)
        y_pred = [self.classes_[i] if v &gt; self.no_class_threshold else self.unknown_class
                  for i, v in zip(argmax(preds, axis=1), max(preds, axis=1))]
        return y_pred

Just use this class as you were using the DecisionTreeClassifier before. You can pass all the parameters from DecisionTreeClassifier plus an extra one: no_class_threshold.
The no_class_threshold works in the following way:
IF
    prediction_probability &gt; no_class_threshold
THEN
    output predicted class
ELSE
    output 'unknown'

Changing classifier
You might not want to stay with the DecisionsTreeClassfier, and instead, you might want to experiment with other classifiers. You can do this by inheriting from another Scikit-learn class. Simple as that!
For example:
from sklearn.svm import SVC

class MyClassifier(SVC):
    [...]

Choosing the no_class_threshold
You can:

Choose it manually, to a degree that you feel confident
Use hyper-parameter tuning to learn it, if you have data points labelled as 'unknown'.

Example
Here is a script you can run to test all the above.
from numpy import argmax, max
from sklearn.svm import SVC


class MyClassifier(SVC):
    unknown_class = 'unknown'

    def __init__(self, no_class_threshold=0.75, **kwargs):
        self.no_class_threshold = no_class_threshold
        super().__init__(**kwargs)

    def predict(self, X):
        preds = self.predict_proba(X)
        y_pred = [self.classes_[i] if v &gt; self.no_class_threshold else self.unknown_class
                  for i, v in zip(argmax(preds, axis=1), max(preds, axis=1))]
        return y_pred


if __name__ == '__main__':
    X = [[1, 1, 1],
         [1, 0, 1],
         [0, 0, 0],
         [0, 1, 0],
         [0, 1, 1]]
    y = ['class1', 'class1', 'class0', 'class0', 'class0']

    clf = MyClassifier(no_class_threshold=0.55, probability=True, C=1)

    clf.fit(X, y)

    X2 = [[1, 1, 0],
          [0, 1, 1],
          [0, 0, 1]]
    pred = clf.predict(X2)
    print(pred)