title: Add Placeholder to layer
body: <p>I have a Tensorflow layer with 2 nodes. These are the output nodes of another 2 larger hidden layers. Now I want to add 2 new nodes to this layer, so I end up with 4 nodes in total, and do some last computation. The added nodes are implemented as Placeholders so far, and have a dynamic shape depending on the batch size. Here is a sketch of the net:</p>
  
  <p><a href="https://i.stack.imgur.com/EFdki.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/EFdki.png" alt="Nodes 3 and 4 have to be concatenated to nodes 1 and 2 from the middle layer"></a></p>
  
  <p>Now I want to concatenate Nodes 3 and 4 to the nodes 1 and 2 of the previously computed layer. I know there is <code>tf.concat</code> for this, but I don't understand how to do this correctly.</p>
  
  <p>How do I add Placeholders of the same batchsize as the original net input to a specific layer?</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>When I use <code>tf.concat</code> over <code>axis=1</code>, I end up with the following problem:</p>
  
  <pre><code>z = tf.placeholder(tf.float32, shape=[None, 2])
  Weight_matrix = weight_variable([4, 2])
  bias = bias_variable([4, 2])
  concat = tf.concat((dnn_out, z), 1)
  h_fc3 = tf.nn.relu(tf.matmul(concat, Weight_matrix) + bias)
  </code></pre>
  
  <p>Adding the bias to the <code>tf.matmul</code> result throws an <code>InvalidArgumentError: Incompatible shapes: [20,2] vs. [4,2]</code>.</p>