Training multiple keras models and combining outputs to determine losses I'm trying to predict the future states of a 1D travelling wave (square, triangle and sawtooth) using a deep learning setup in Keras. The waves are discretised in a 1024 data points. As this gives a quite high input dimensionality for an RNN, the 1024 inputs are reduced to a low dimensional latent space of 10 dimensions using a convolutional autoencoder setup. The RNN is given 5 previous time step latent inputs to predict the next time step. I have trained the autoencoder and RNN seperately and obtained pretty decent results, but ideally could be improved. The following figure gives an example of how I trained the models in a decoupled manner.



My idea to improve this model by coupling the training process. Here the autoencoder is trained with the same loss as the RNN. The following picture gives an idea of what i'm trying to achieve.



My hope is that the latent space will be more suitable for time predictions using this setup, as the autoencoder is penalised for creating a latent space which is hard to do time predictions with.

My problem is that I don't know how to implement the combining of the models in this manner in Keras. Could anyone help me with this?

P.S.

I have asked another question about this topic in the following link which gives some examples of my project and on the decoupled training results.

Other question on Data Science Stack Exchange
In keras a model can have multiple outputs, for example:

model = Model(inputs=[a1, a2], outputs=[b1, b2, b3])


Aand if you have multiple outputs you can have multiple losses, for example:

model.compile(loss=[loss1, loss2, loss3], optimizer='sgd')


By doing this, when you train and backpropagate your model you are using all the losses to train your model since you are minimizing all the losses. In this way you can couple your two models as you suggested.

Additionally compile has the parameter loss_weights in which you can change the weights of the different losses.