title: TensorFlow Training
body: <p>Assuming I have a very simple neural network, like multilayer perceptron. For each layer the activation function is sigmoid and the network are fully connected.</p>
  
  <p>In TensorFlow this might be defined like this:</p>
  
  <pre class="lang-py prettyprint-override"><code>    sess = tf.InteractiveSession()
  
      # Training Tensor
      x = tf.placeholder(tf.float32, shape = [None, n_fft])
      # Label Tensor
      y_ = tf.placeholder(tf.float32, shape = [None, n_fft])
  
      # Declaring variable buffer for weights W and bias b
      # Layer structure [n_fft, n_fft, n_fft, n_fft]
      # Input -&gt; Layer 1
      struct_w = [n_fft, n_fft]
      struct_b = [n_fft]
      W1 = weight_variable(struct_w, 'W1')
      b1 = bias_variable(struct_b, 'b1')
      h1 = tf.nn.sigmoid(tf.matmul(x, W1) + b1)
  
      # Layer1 -&gt; Layer 2
      W2 = weight_variable(struct_w, 'W2')
      b2 = bias_variable(struct_b, 'b2')
      h2 = tf.nn.sigmoid(tf.matmul(h1, W2) + b2)
  
      # Layer2 -&gt; output
      W3 = weight_variable(struct_w, 'W3')
      b3 = bias_variable(struct_b, 'b3')
      y = tf.nn.sigmoid(tf.matmul(h2, W3) + b3)
  
      # Calculating difference between label and output using mean square error
      mse = tf.reduce_mean(tf.square(y - y_))
  
      # Train the Model
      # Gradient Descent
      train_step = tf.train.GradientDescentOptimizer(0.3).minimize(mse)
  </code></pre>
  
  <p>The design target for this model is to map a <code>n_fft</code> points fft spectrogram to another <code>n_fft</code> target spectrogram. Let's assume both the training data and target data are of size <code>[3000, n_fft]</code>. They are stored in variables <code>spec_train</code> and <code>spec_target</code>.</p>
  
  <p>Now here comes the question. For TensorFlow is there any difference between these two trainings?</p>
  
  <p>Training 1:</p>
  
  <pre class="lang-py prettyprint-override"><code>for i in xrange(200):
          train_step.run(feed_dict = {x: spec_train, y_: spec_target})
  </code></pre>
  
  <p>Training 2:</p>
  
  <pre class="lang-py prettyprint-override"><code>for i in xrange(200):
          for j in xrange(3000):
              train = spec_train[j, :].reshape(1, n_fft)
              label = spec_target[j, :].reshape(1, n_fft)
              train_step.run(feed_dict = {x: train, y_: label})
  </code></pre>
  
  <p>Thank you very much!</p>