title: Tensorflow returns same weights in hidden layer
body: <p>I'm trying to train a model that will emulate a sine function.
  Using 3 fully connected layers with 1-15-1 neurons. So I have 2 weight variables, 1x15 and 15x1. After training they have the same values:</p>
  
  <pre><code>[[0.02432664] [0.02432664] [0.02432664] [0.02432664] [0.02432664] 
  [0.02432664] [0.02432664] [0.02432664] [0.02432664] [0.02432664] 
  [0.02432664] [0.02432664] [0.02432665] [0.02432665] [0.02432665]]
  
  [[0.02432664 0.02432664 0.02432664 0.02432664 0.02432664 0.02432664
    0.02432664 0.02432664 0.02432664 0.02432664 0.02432664 0.02432664
    0.02432665 0.02432665 0.02432665]]
  </code></pre>
  
  <p>Here is my code:</p>
  
  <pre><code>W1 = tf.Variable(tf.ones([15, 1]) * 0.3)
  W2 = tf.Variable(tf.ones([1, 15]) * 0.3)
  label = tf.placeholder(dtype=tf.float32, shape=[1, None])
  result = tf.placeholder(dtype=tf.float32)
  
  l1 = tf.matmul(W1, label)
  l2 = tf.matmul(W2, l1)
  loss = tf.square(l2 - result)
  loss = tf.reduce_sum(loss)
  train_op = tf.train.AdamOptimizer().minimize(loss)
  
  
  sess = tf.Session()
  sess.run(tf.global_variables_initializer())
  feed_dict = {label: [labels], result: values}
  
  for i in range(0, 500):
      sess.run(train_op, feed_dict)
  </code></pre>
  
  <p>Values and labels from feed_dict look like this:</p>
  
  <pre><code>[0.041410759078128294, 0.5139904997395468, 0.97, 0.9710565162951535, 
   0.6471067811865476, 0.20643446504023055]
  
  [1, 15, 50, 60, 75, 95]
  </code></pre>
  
  <p>(Range 1-100 equivalent to 0 - PI)</p>
  
  <p>Have no idea why tensorflow is doing so. Can you suggest what has gone wrong and how to fix it?</p>