title: Tensorflow weights stay at nan
body: <p>I am trying to code a basic logistic regression in tensorflow ( full code here <a href="http://pastebin.com/m6EwakjL" rel="nofollow">http://pastebin.com/m6EwakjL</a>) using my own data. The problem is that foreach batch run, all values in my weight  stay at nan which causes all other calculations ( cost, etc ) to ( obviously) fail. </p>
  
  <p>I tried to initialize my weight matrix to 'random_normal' or 'zeros' but this doesn't make any difference:</p>
  
  <pre><code># neither work
  batch_size = 100
  W = tf.Variable(tf.random_normal([batch_size,num_features]))
  W = tf.Variable(tf.zeros([batch_size,num_features]))
  </code></pre>
  
  <p>the below code is for the LR setup.</p>
  
  <pre><code># Xtrain = 100k x 10 , 10 features
  # Ytrain = 100k x  2, 2 classes
  num_features = Xtrain.shape[1]
  num_classes = Ytrain.shape[1]
  
  # tf Graph Input
  x = tf.placeholder(tf.float32, [batch_size, num_features])
  y = tf.placeholder(tf.float32, [batch_size, num_classes])
  
  # Set model weights
  W = tf.Variable(tf.random_normal([num_features, num_classes]))
  b = tf.Variable(tf.zeros([num_classes]))
  
  mm = tf.matmul(x, W)
  model = mm + b
  
  # Construct model
  pred = tf.nn.softmax(model)
  
  # Minimize error using cross entropy
  #cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=1))
  cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred)))
  # Gradient Descent
  optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
  </code></pre>
  
  <p>Below I try to view the value(s) of W, x and other vars for debugging purposes. But my W is always 'nan' .</p>
  
  <pre><code># Initializing the variables
  init = tf.initialize_all_variables()
  
  # Launch the graph
  with tf.Session() as sess:
      sess.run(init)
  
      # Training cycle
      for epoch in range(training_epochs):
          avg_cost = 0.
          total_batch = int(Xtrain.shape[0]/batch_size)
          # Loop over all batches
          for i in range(total_batch):
              #s = (i - 1) * batch_size
              s = i * batch_size
              e = s + batch_size 
              batch_xs = Xtrain[s:e,:]
              batch_ys = Ytrain[s:e,:]
              #print (" i = %s, s = %s, e = %s"  % (i,s,e))
              # Fit training using batch data
              _, c, m, matrix_multiply, bias, w = sess.run([optimizer, cost,model,mm,b,W], feed_dict={x: batch_xs,y: batch_ys})
              print ("i = %s, start = %s, end = %s" % (i,s,e))
              print ("# X shape #")
              print (batch_xs.shape)
              print ("w = %s" % w)
              print ("matrix_multiply = %s" % matrix_multiply)
              print ("bias = %s" % bias) 
              print ("model = %s" % m)
  </code></pre>
  
  <p>Any thoughts on way my w matrix is always nan ?</p>