title: Why do these two ways of doing an embedding lookup give a different answer?
body: <p>Suppose I have the following matrix:</p>
  
  <p><a href="https://i.stack.imgur.com/h5gtU.gif" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/h5gtU.gif" alt="enter image description here"></a></p>
  
  <p>So, if I want to lookup the second row, I should get [3 4].</p>
  
  <p>This is how I implemented the lookup mechanism using tensorflow operations:</p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  
  from tensorflow.python.estimator.model_fn import EstimatorSpec
  
  
  def model_fn_1(features, labels, mode):
      x = tf.constant([[1]])
      labels = tf.constant([[10.]])
  
      with tf.name_scope('Embedding_Layer'):
          m = np.array([[1, 2], [3, 4]], np.float32)
  
      lookup = tf.nn.embedding_lookup(m, x, name='embedding_matrix_1')
      lookup = tf.Print(lookup, [lookup])
  
      preds = tf.keras.layers.Dense(1)(lookup)
      loss = tf.reduce_mean(labels - preds)
      train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss, tf.train.get_global_step())
  
      eval_metric_ops = {'accuracy': tf.metrics.accuracy(labels, preds)}
      return EstimatorSpec(mode=mode, loss=loss, train_op=train_op, eval_metric_ops=eval_metric_ops)
  
  
  model_1 = tf.estimator.Estimator(model_fn_1)
  model_1.train(input_fn=lambda: None, steps=1)
  </code></pre>
  
  <p>As expected, the output from <code>lookup</code> while training is:</p>
  
  <pre><code>2017-11-08 21:17:49.010728: I C:\tf_jenkins\home\workspace\rel-win\M\windows\PY\36\tensorflow\core\kernels\logging_ops.cc:79] [[[3 4]]]
  </code></pre>
  
  <p>The problem is I do not get the same output when i try to implement the embedding lookup using Keras layers:</p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  from tensorflow.python.estimator.model_fn import EstimatorSpec
  
  
  def model_fn(features, labels, mode):
      x = tf.constant([[1]])
      labels = tf.constant([[10.]])
  
      m = np.array([[1, 2], [3, 4]], np.float32)
      with tf.name_scope('Embedding_Layer'):
          n = tf.keras.layers.Embedding(2, 2, weights=[m], input_length=1, name='embedding_matrix_1', trainable=False)
  
      lookup = n(x)
      lookup = tf.Print(lookup, [lookup])
  
      preds = tf.keras.layers.Dense(1)(lookup)
      loss = tf.reduce_mean(labels - preds)
      train_op = tf.train.GradientDescentOptimizer(0.01).minimize(loss, tf.train.get_global_step())
  
      eval_metric_ops = {'accuracy': tf.metrics.accuracy(labels, preds)}
      return EstimatorSpec(mode=mode, loss=loss, train_op=train_op, eval_metric_ops=eval_metric_ops)
  
  
  model = tf.estimator.Estimator(model_fn)
  model.train(input_fn=lambda: None, steps=1)
  </code></pre>
  
  <p>The output of <code>lookup</code> is some random numbers like:</p>
  
  <pre><code>2017-11-08 21:20:59.046951: I C:\tf_jenkins\home\workspace\rel-win\M\windows\PY\36\tensorflow\core\kernels\logging_ops.cc:79] [[[0.532017469 0.885832787]]]
  </code></pre>
  
  <p>In my mind, the Keras implementation is identical to the tensforlow implementation. Why am I not getting the same output and how do I fix the Keras implementation?</p>