How to combine two logistic regression models trained on different set of data? My data has a hierarchy structure - meaning that there is an N class at level 1 and an M class at level M. After training both models separately with a different set of data (both are Logistic regression but can be changed) I want to predict to N + M classes.
Currently, I am solving it by collecting all data before training and adding examples from level 2 to level 1 results in (N + 1) classes (+ 1 meaning all examples from level 2) and if the model at level 1 predicts that &quot;+1&quot; class then a send the query to the second model. But collecting the examples before training is costly and also the training set is unbalanced after collecting. Pseudocode:
clf1 = LogisticRegression()
clf1.fit(X1 + X2, Y1 + [&quot;ID of +1 class&quot;] * len(Y2))
clf2 = LogisticRegression()
clf2.fit(X2, Y2)
if cl1.predict(X_test_example) == [&quot;ID of +1 class&quot;]:
    clf2.precict(X_test_example)


But I need to preserve this hierarchical structure so the merging the two models directly is not possible. Is there a way to combine two outputs of classification models (in a form of the probability distribution) to get meaningful output are avoid problems with the overconfidence of models? Pseudocode:
clf1 = LogisticRegression()
clf1.fit(X1, Y1)
clf2 = LogisticRegression()
clf2.fit(X2, Y2)
combined_clf = clf1 + clf2
I can think of two options, there are probably more:

Train a binary classification &quot;top model&quot; which predicts whether the instance belongs to  model 1 or model 2. The two final models only need to be trained with their respective data and only predict their respective N or M classes. The performance of the top model is crucial in this option.
Train an open set classification model. It's a bit similar to what you're doing currently by adding a class &quot;other model&quot; to the first model, except that &quot;other&quot; is not a regular class and you don't need to train the model with instances from this class. This is related to one-class classification and requires a specific training method which can handle open-set classification.