title: LSTM Loss remains the same after numerous iterations
body: <p>Thanks for looking into this question! :)</p>
  
  <p>I attempted to train an LSTM network to predict next 10-day stock prices of Google based on past 30-day stock prices. I trained the LSTM but the loss barely reduced even after 200 iterations. I suspected that the issue might be due to the feed_dict in tf Session. However, I have not identified any issue with that (perhaps due to my superficial knowledge). It seems that the optimizer refreshes every iterations in the tf Session. </p>
  
  <p>Would appreciate if I could seek advice on what might have gone wrong in the code, if my understanding on the usage of Optimizer has been wrong.</p>
  
  <p>Thanks for your help!!</p>
  
  <pre><code>import pandas as pd
  import matplotlib.pyplot as plt
  import numpy as np
  import sys
  import csv
  import random
  import tensorflow as tf
  from tensorflow.contrib import rnn
  
  # Define data reader
  def read_data(fname):
      with open(fname) as f:
          data = list(csv.reader(f))
          d_mat = np.array(data)
          d_trsp = np.transpose(d_mat)
  
          date = np.transpose(d_trsp[0])
          p_open = d_trsp[1]
          vol = d_trsp[6]
          chg = d_trsp[7]
          chg = [float(i) for i in chg]
  
          return vol, chg
  
  vol, training_data = read_data('GOOGL.csv')
  
  training_data = training_data[0:300]
  print("Loading training data..")
  
  #Split data for learning
  ratio_train = 0.70
  ratio_valid = 0.90-ratio_train
  ratio_test = 0.10 #fixed at 10% of dataset
  
  # Parameters
  learning_rate = 0.005
  training_iters = 100
  display_step = 1
  x_size = 30
  y_size = 5
  n_hidden = 256
  
  # Variables
  x = tf.placeholder("float", [265, x_size])
  y = tf.placeholder("float", [265, y_size])
  
  weights = {
      'out': tf.Variable(tf.random_normal([n_hidden, y_size]))
      }
  
  biases = {
      'out': tf.Variable(tf.random_normal([y_size]))
      }
  
  # Preprocess Data
  def prod_data(data):
      x = []
      y = []
  
      iter = len(data)-x_size-y_size
      for i in range(0, iter):
          x.append(data[i:i+x_size])
          y.append(data[i+x_size+1: i+x_size+1+y_size])
  
      return x, y
  
  a,b = prod_data(training_data)
  
  # Define RNN architecture
  def RNN(x, weights, biases):
      # Reshape x to [1, n_input]
      x = tf.reshape(x, [-1, x_size])
      x = tf.split(x, x_size, 1)
  
      rnn_cell = rnn.MultiRNNCell([rnn.BasicLSTMCell(n_hidden), rnn.BasicLSTMCell(n_hidden)])
      outputs, states = rnn.static_rnn(rnn_cell, x, dtype = tf.float32)
  
      return tf.matmul(outputs[-1], weights['out'] + biases['out'])
  
  pred = RNN(x, weights, biases)
  
  # Loss and Optimizer
  cost = tf.reduce_mean((pred-y)**2)
  optimizer = tf.train.RMSPropOptimizer(learning_rate = learning_rate).minimize(cost)
  
  # Initialization
  init = tf.global_variables_initializer()
  
  # Launch Tensor graph
  with tf.Session() as sess:
      sess.run(init)
      step = 0
      loss_total = 0
      loss_coll = []
      end_offset = len(training_data)-y_size-x_size-1  
  
      while step &lt; training_iters:
  
          _, loss, model_pred = sess.run([optimizer, cost, pred], \
                                                  feed_dict={x: a, y: b})
  
          # Update total loss and accuracy
          loss_total += loss
          loss_coll.append(loss)
  
  
          if (step+1) % display_step == 0:
              print("Loss at step " + str(step) + " = " + str(loss))
              loss_total = 0
  
          step += 1
  
      print("Optimization Finished!")
  </code></pre>