Tricky stacking models in keras I'm trying to write a model with keras, that is built as shown below:
  |                 +-----+
  +-&gt;+------+       |     |
+---&gt;|  NN  |------&gt;|     |
|    +------+       |     |
| |                 |     |
| +-&gt;+------+       |     |
+---&gt;|  NN  |------&gt;|  L  |
|    +------+       |  S  |------+-----&gt; Output value
|                   |  T  |      |
|      ...          |  M  |      |
| |                 |     |      |
| +-&gt;+------+       |     |      |
+---&gt;|  NN  |------&gt;|     |      |
|    +------+       |     |      |
|                   +-----+      |
|    +-------+                   |
+----| Delay |-------------------+
     +-------+

I have several simple sequential models (marked as NN), that receive two numerical value as input, they calculate some other numeric values (one per each network). These values are passed to LSTM network, which produces a single value as an output and this value additionaly is passed to initial networks (possibly with some delay) as one of two inputs. I work with time series, so calculated final value is passed to network alongside with the next time series value.
I use LSTM to store some &quot;state&quot;. It is not quite difficult to build separate &quot;sub-models&quot;, but I don't realize, how to join them together as I need, i.e. how to make a final output to be passed to initial networks and how to stack them in the described way (not as a chain).
What I've found: I found keras.layers.Concatenate, but it seems not to be what I'm looking for... But maybe (I hope) I'm mistaken.
You can use the below type code.
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, LSTM, Concatenate
from tensorflow.keras.models import Model

# input of first NN
input_l1 = Input(shape=(2,))
out_l1 = Dense(1)(input_l1)

# input is 2nd NN
input_l2 = Input(shape=(2,))
out_l2 = Dense(1)(input_l2)

# concat layer output shape will be (None, 2) becuase we concatinated 2 dense layer outputs/ 
concat_vec = Concatenate()([out_l1, out_l2])

# we need 3d input to LSTM i.e. ( Batch_Size, no of time steps, feature space)
# We have 2 inputs so expanded dim to (None, 2, 1)
expanded_concat = tf.expand_dims(concat_vec, axis=2)

# LSTM
lstm_out = LSTM(15)(expanded_concat)

model = Model(inputs=[input_l1, input_l2], outputs=lstm_out)