title: How do I share weights across different RNN cells that feed in different inputs in Tensorflow?
body: <p>I'm curious if there is a good way to share weights across different RNN cells while still feeding each cell different inputs.</p>
  
  <p>The graph that I am trying to build is like this:</p>
  
  <p><a href="https://i.stack.imgur.com/k4OeI.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/k4OeI.png" alt="enter image description here"></a></p>
  
  <p>where there are three LSTM Cells in orange which operate in parallel and between which I would like to share the weights.</p>
  
  <p>I've managed to implement something similar to what I want using a placeholder (see below for code). However, using a placeholder breaks the gradient calculations of the optimizer and doesn't train anything past the point where I use the placeholder. Is it possible to do this a better way in Tensorflow?</p>
  
  <p>I'm using Tensorflow 1.2 and python 3.5 in an Anaconda environment on Windows 7.</p>
  
  <p>Code:</p>
  
  <pre><code>def ann_model(cls,data, act=tf.nn.relu):
      with tf.name_scope('ANN'):
          with tf.name_scope('ann_weights'):
              ann_weights = tf.Variable(tf.random_normal([1,
                                                          cls.n_ann_nodes]))
          with tf.name_scope('ann_bias'):
              ann_biases = tf.Variable(tf.random_normal([1]))
          out = act(tf.matmul(data,ann_weights) + ann_biases)
      return out
  
  def rnn_lower_model(cls,data):
      with tf.name_scope('RNN_Model'):
          data_tens = tf.split(data, cls.sequence_length,1)
          for i in range(len(data_tens)):
              data_tens[i] = tf.reshape(data_tens[i],[cls.batch_size,
                                                       cls.n_rnn_inputs])
  
          rnn_cell = tf.nn.rnn_cell.BasicLSTMCell(cls.n_rnn_nodes_lower)
  
          outputs, states = tf.contrib.rnn.static_rnn(rnn_cell,
                                                      data_tens,
                                                      dtype=tf.float32)
  
          with tf.name_scope('RNN_out_weights'):
              out_weights = tf.Variable(
                      tf.random_normal([cls.n_rnn_nodes_lower,1]))
          with tf.name_scope('RNN_out_biases'):
              out_biases = tf.Variable(tf.random_normal([1]))
  
          #Encode the output of the RNN into one estimate per entry in 
          #the input sequence
          predict_list = []
          for i in range(cls.sequence_length):
              predict_list.append(tf.matmul(outputs[i],
                                            out_weights) 
                                            + out_biases)
      return predict_list
  
  def create_graph(cls,sess):
      #Initializes the graph
      with tf.name_scope('input'):
          cls.x = tf.placeholder('float',[cls.batch_size,
                                         cls.sequence_length,
                                         cls.n_inputs])
      with tf.name_scope('labels'):
          cls.y = tf.placeholder('float',[cls.batch_size,1])
      with tf.name_scope('community_id'):
          cls.c = tf.placeholder('float',[cls.batch_size,1])
  
      #Define Placeholder to provide variable input into the 
      #RNNs with shared weights    
      cls.input_place = tf.placeholder('float',[cls.batch_size,
                                                cls.sequence_length,
                                                cls.n_rnn_inputs])
  
      #global step used in optimizer
      global_step = tf.Variable(0,trainable = False)
  
      #Create ANN
      ann_output = cls.ann_model(cls.c)
      #Combine output of ANN with other input data x
      ann_out_seq = tf.reshape(tf.concat([ann_output for _ in 
                                              range(cls.sequence_length)],1),
                              [cls.batch_size,
                               cls.sequence_length,
                               cls.n_ann_nodes])
      cls.rnn_input = tf.concat([ann_out_seq,cls.x],2)
  
      #Create 'unrolled' RNN by creating sequence_length many RNN Cells that
      #share the same weights.
      with tf.variable_scope('Lower_RNNs'):
          #Create RNNs
          daily_prediction, daily_prediction1 =[cls.rnn_lower_model(cls.input_place)]*2
  </code></pre>
  
  <p>When training mini-batches are calculated in two steps:</p>
  
  <pre><code>RNNinput = sess.run(cls.rnn_input,feed_dict = {
                                              cls.x:batch_x,
                                              cls.y:batch_y,
                                              cls.c:batch_c})
  _ = sess.run(cls.optimizer, feed_dict={cls.input_place:RNNinput,
                                         cls.y:batch_y,
                                         cls.x:batch_x,
                                         cls.c:batch_c})
  </code></pre>
  
  <p>Thanks for your help. Any ideas would be appreciated.</p>