title: Tensor Objects are not iterable when eager execution... while using Keras shape function
body: <p>I am trying to implement the following. I need help with the error, I get, during execution, and in general, suggestions, if there is a better way to implement the same idea, described below.</p>
  
  <p>Network Model - 
  <a href="https://i.imgur.com/LdKwXRP.png" rel="nofollow noreferrer">https://i.imgur.com/LdKwXRP.png</a></p>
  
  <p>I have a variable number of networks, e.g. in this fig, there are 3 illustrated. Each layer is executed sequentially.</p>
  
  <p>The outputs of the other dense layers are used as an input for doing a dot product with the output of the dense layer that is currently being executed.
  The implementation for that is done by, storing the output of these intermediate layers, in a class variable list, called ActorNetwork.k_list.</p>
  
  <p>During implementation, I get the error - </p>
  
  <pre><code>Traceback (most recent call last):
    File "training-code.py", line 80, in &lt;module&gt;
      main(args)
    File "training-code.py", line 36, in main
      ActorNetwork(sess, observation_dim[i], action_dim[i], float(args['actor_lr']), float(args['tau']), n))
    File "/home/rangwala/maddpg-attn/actorcriticv2.py", line 30, in __init__
      self.mainModel, self.mainModel_weights, self.mainModel_state = self._build_model()
    File "/home/rangwala/maddpg-attn/actorcriticv2.py", line 55, in _build_model
      keys = Input(shape=(K.shape(ActorNetwork.k_list,)))
    File "/home/rangwala/anaconda3/envs/comm-nav-cpu/lib/python3.6/site-packages/keras/engine/input_layer.py", line 171, in Input
      batch_shape = (None,) + tuple(shape)
    File "/home/rangwala/anaconda3/envs/comm-nav-cpu/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 431, in __iter__
      "Tensor objects are not iterable when eager execution is not "
  TypeError: Tensor objects are not iterable when eager execution is not enabled. To iterate over this tensor use tf.map_fn.
  </code></pre>
  
  <p>CODE - </p>
  
  <pre><code>class ActorNetwork(object):
  """
  Implements actor network
  """
  
      k_list = [] #stores the values from the intermediate layers
  
  # More code in between
  
      def _build_model(self):
          input_obs = Input(shape=(self.state_dim,))
          keys = Input(shape=(K.shape(ActorNetwork.k_list, ))) 
          #k_list is a class variable, of the ActorNetwork class.
  
          h = Dense(400)(input_obs)
          h = Activation('relu')(h)
  
          query = Dense(self.n_attn, name="keys_layer")(h)
          ky_list = ActorNetwork.k_list
          keys_list = ky_list.pop(self.n) #remove own entry from the list, for dot product
          concat_layer = Concatenate(axis=1)
          all_agents = concat_layer(keys_list)
  
          attn = tf.einsum('i, i-&gt;ij', [query, all_agents]) / self.temper #dot product
  
          attn = Activation('softmax')(attn)
          attn = Dropout(0.1)(attn)
  
          attn_out = tf.einsum('ik, k-&gt;i', [all_agents, attn])
          attn_add = Lambda(lambda x: x[0] + x[1])([query, attn_out]) #add own value to dot product value
  
          h = Dense(self.action_dim)(attn_add)
          pred = Activation('tanh')(h)
          pred = BatchNormalization()(pred)
  
          model = Model(inputs=[input_obs, keys], outputs=pred)
          model.compile(optimizer='Adam', loss='categorical_crossentropy')
  
          attn_layer_out = model.get_layer("keys_layer").output
          ActorNetwork.k_list[self.n] = attn_layer_out 
  
          return model, model.trainable_weights, input_obs
  </code></pre>