title: tensorflow loss is nan while training an RNN
body: <p>Running an RNN with a single GRU cell, I'm running into situations where I get the following stacktrace</p>
  
  <pre><code>Traceback (most recent call last):
    File "language_model_test.py", line 15, in &lt;module&gt;
      test_model()
    File "language_model_test.py", line 12, in test_model
      model.train(random_data, s)
    File "/home/language_model/language_model.py", line 120, in train
      train_pp = self._run_epoch(data, sess, inputs, rnn_ouputs, loss, trainOp, verbose)
    File "/home/language_model/language_model.py", line 92, in _run_epoch
      loss, _= sess.run([loss, trainOp], feed_dict=feed)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 767, in run
      run_metadata_ptr)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 952, in _run
      fetch_handler = _FetchHandler(self._graph, fetches, feed_dict_string)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 408, in __init__
      self._fetch_mapper = _FetchMapper.for_fetch(fetches)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 230, in for_fetch
      return _ListFetchMapper(fetch)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 337, in __init__
      self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 238, in for_fetch
      return _ElementFetchMapper(fetches, contraction_fn)
    File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 271, in __init__
      % (fetch, type(fetch), str(e)))
  TypeError: Fetch argument nan has invalid type &lt;type 'numpy.float32'&gt;, must be a string or Tensor. (Can not convert a float32 into a Tensor or Operation.)
  </code></pre>
  
  <p>The step for calculating loss appears to be the issue</p>
  
  <pre><code>def train(self,data, session=tf.Session(), verbose=10):
  
          print "initializing model"
          self._add_placeholders()
          inputs = self._add_embedding()
          rnn_ouputs, _ = self._run_rnn(inputs)
          outputs = self._projection_layer(rnn_ouputs)
          loss = self._compute_loss(outputs)
          trainOp = self._add_train_step(loss)
          start = tf.initialize_all_variables()
          saver = tf.train.Saver()
  
          with session as sess:
              sess.run(start)
  
              for epoch in xrange(self._max_epochs):
                  train_pp = self._run_epoch(data, sess, inputs, rnn_ouputs, loss, trainOp, verbose)
                  print "Training preplexity for batch {} - {}".format(epoch, train_pp)
  </code></pre>
  
  <p>Here is the code for <code>_run_epoch</code></p>
  
  <p>Any place with loss comes back <code>nan</code></p>
  
  <pre><code>def _run_epoch(self, data, session, inputs, rnn_ouputs, loss, trainOp, verbose=10):
      with session.as_default() as sess:
          total_steps = sum(1 for x in data_iterator(data, self._batch_size, self._max_steps))
          train_loss = []
          for step, (x,y, l) in enumerate(data_iterator(data, self._batch_size, self._max_steps)):
              print "step - {0}".format(step)
              feed = {
                  self.input_placeholder: x,
                  self.label_placeholder: y,
                  self.sequence_length: l,
                  self._dropout_placeholder: self._dropout,
              }
              loss, _= sess.run([loss, trainOp], feed_dict=feed)
              print "loss - {0}".format(loss)
              train_loss.append(loss)
              if verbose and step % verbose == 0:
                  sys.stdout.write('\r{} / {} : pp = {}'. format(step, total_steps, np.exp(np.mean(train_loss))))
                  sys.stdout.flush()
              if verbose:
                  sys.stdout.write('\r')
  
          return np.exp(np.mean(train_loss))
  </code></pre>
  
  <p>This is arising when I test my code by using the following for my data
  <code>random_data = np.random.normal(0, 100, size=[42068, 46])</code> which is designed to mimic using word ids to pass around as inputs. The rest of my code can be found in the following <a href="https://gist.github.com/IoDmitri/499b8273d4504bec0ace61f39ce1a7c3" rel="nofollow noreferrer">gist</a></p>
  
  <p><strong>EDIT</strong> Here is the way that I run the test suit where this issue arises:</p>
  
  <pre><code>def test_model():
      model = Language_model(vocab=range(0,101))
      s = tf.Session()
      #1 more than step size to acoomodate for the &lt;eos&gt; token at the end
      random_data = np.random.normal(0, 100, size=[42068, 46])
      # file = "./data/ptb.test.txt"
      print "Fitting started"
      model.train(random_data, s)
  
  if __name__ == "__main__":
      test_model() 
  </code></pre>
  
  <p>If I substitute <code>random_data</code> into other language models, they will also output <code>nan</code> for the cost. My understanding is that tensorflow, via passing in the feed-dict, should be taking the numeric values and retrieving the appropriate embedding vector corresponding to the id, I'm not understand why the <code>random_data</code> is causing <code>nan</code>'s for other models. </p>