Deep neural network models merging Recently I am working on the neural network deep learning algorithms, just curious to ask is it possible to merge two neural network models and to output one model that contains all the learned knowledge from the two models?
For example:
Model A can work on feature [A, B]
Model B can work on feature [C, D]
(Model type is the same for model A and model B if this makes things easier.)
After model merging, we get model C which is capable of working on feature [A, B, C, D].
I have looked into the transfer learning and Siamese network, but to my knowledge, I don't think these techniques can help me achieve my goal? (please correct me if I am wrong).
So any ideas will be much appreciated for merging the two models, or any possible technique, terminologies are welcomed.
The concept of Ensemble Learning is very applicable to your scenario: given two machine learning models that can predict well (or even just better than a random choice), combine the outputs of both models (with another ML model) to produce a unified model (let's call it Model C) that is more predictive than the individual models alone.
A straightforward application to your scenario could involve using the outputs of Models A and B along with some additional features from the original feature set as inputs to Model C. This larger set of features would allow for model C to learn for example when to the signal from Model A is more informative than B based on the current context provided by additional features from the original set.
It may also be possible to do this within a deep learning framework by adding both models into the same graph and adding a layer that ensembles the signals from the two models for an output prediction.