title: Huge loss in regression exercise! Testing with batch input?
body: <p>I am currently running the following code in order to predict the price of a house based on 6 parameters:</p>
  
  <pre><code>import pandas as pd
  import tensorflow as tf
  import numpy as np
  
  housing = pd.read_csv('cal_housing_clean.csv')
  
  X = housing.iloc[:,0:6]
  y = housing.iloc[:,6:]
  
  from sklearn.model_selection import train_test_split
  
  X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.3)
  
  from sklearn.preprocessing import MinMaxScaler
  scaler = MinMaxScaler()
  scaler.fit(X_train)
  X_train = pd.DataFrame(data=scaler.transform(X_train),columns = X_train.columns,index=X_train.index)
  X_test = pd.DataFrame(data=scaler.transform(X_test),columns = X_test.columns,index=X_test.index)
  
  X_data = tf.placeholder(dtype = "float", shape=[None,6])
  y_target = tf.placeholder(dtype = "float", shape=[None,1])
  
  hidden_layer_nodes = 10
  
  w1 = tf.Variable(tf.random_normal(shape=[6,hidden_layer_nodes]))
  b1 = tf.Variable(tf.random_normal(shape=[hidden_layer_nodes]))
  w2 = tf.Variable(tf.random_normal(shape=[hidden_layer_nodes,1]))
  b2 = tf.Variable(tf.random_normal(shape=[1]))
  
  hidden_output = tf.nn.relu(tf.add(tf.matmul(X_data,w1),b1))
  y_output = tf.add(tf.matmul(hidden_output,w2),b2)
  
  loss = tf.reduce_mean(tf.square(y_target-y_output))
  
  optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.00001)
  train = optimizer.minimize(loss)
  
  init = tf.global_variables_initializer()
  
  steps = 100000
  
  with tf.Session() as sess:
  
      sess.run(init)
  
      for i in range(steps):
  
          sess.run(train, feed_dict={X_data:X_train,y_target:y_train})
  
          if i%500 == 0:
  
              print('Currently on step {}'.format(i))
  
              training_cost = sess.run(loss, feed_dict={X_data:X_test,y_target:y_test})
              print("Training cost=", training_cost/6192)
  
      training_cost = sess.run(loss, feed_dict={X_data:X_test,y_target:y_test})
      print("Training cost=", training_cost/6192)
  </code></pre>
  
  <p>I hereby thought that since the test_set contains 6192 rows of data, that simply dividing the total loss or error by that value would solve the problem, but unfortunately I reach the following output:</p>
  
  <pre><code>Currently on step 0
  Training cost= 9190063.95866
  Currently on step 500
  Training cost= 9062077.85013
  Currently on step 1000
  Training cost= 8927415.89664
  Currently on step 1500
  Training cost= 8795428.38243
  Currently on step 2000
  Training cost= 8666037.25065
  Currently on step 2500
  Training cost= 8539182.30491
  Currently on step 3000
  Training cost= 8414841.71576
  </code></pre>
  
  <p>in which the error will go down to about 2 Million while I would like to have values close to 100 or 200 thousand.</p>
  
  <p>Maybe there is an error in my code that makes the approximations so bad. I also tried different learning_rates with the same results.</p>
  
  <p>I also wanted to try to test the model by sending in the test data in batches. I tried this:</p>
  
  <pre><code>if i%500 == 0:
  
      rand_ind = np.random.randint(len(X_test),size=8)
  
      feed = {X_data:X_test[rand_ind],y_target:y_test[rand_ind]}
  
      loss = tf.reduce_sum(tf.square(y_target-y_output)) / 8
  
      print(sess.run(loss,feed_dict=feed))
  </code></pre>
  
  <p>but unfortunately I am always told that the index, which I chose with rand_ind it "not in index". </p>