Combining neural networks with different variables My organisation provides consultation to other firms, in part by making use of neural networks trained with extensive datasets that we have collected over the years.

Whenever available, we would inquire for any similar data our clients may have so that we can tailor a new model based on a checkpoint of our own. This has worked well as the data we used for training is relatively generalised and often collected by organisations in general.

However, recently one of our clients provided data that is highly predictive of the same response variable in our own models, but differs wildly in variable types and size.

An example in order to simplify:

I trained my own network to predict height based on weight using 100,000 records. Oftentimes my clients provide an additional 10,000 records of their employees height &amp; weight which allows me to tailor the network to their needs. A new client, however, has provided me with 10,000 records of their employees' gender, country of origin, and daily dairy consumption. All of which, when trained in a separate network, prove highly predictive of height as well.

How do I effectively combine these separate models into a single classifier or regressor?
This process is called fine tuning frequently in the neural network literature. If you already have the trained network, you simply run additional training steps on the new data.

You'll need to balance the weight of the prior training with that of the new observations to some degree, and that decision is parameterized by the number of free variables in the network, the learning rate of the new training steps, and the number of new training steps.

However, your particular case is very complicated by the presence of new variables. Maintaining the form of the prior network is difficult in this case. You may want to simply ensemble a newly trained network on the new data and variables with the old network to combine the predictions.The simplest ensemble (but maybe not the best) would be averaging the results of the two models.