title: LSTM/RNN in pytorch The relation between forward method and training model
body: <p>I'm still fairly new to neural networks, so sorry on beforehand for any ambiguities to the following.</p>
  <p>In a &quot;standard&quot; LSTM implementation for language task, we have the following (sorry for the very rough sketches):</p>
  <pre><code>class LSTM(nn.Module):
      def __init__(*args):
      ...
  
      def forward(self, input, states):
           
          lstn_in = self.model['embed'](input)
          lstm_out, hidden = self.model['lstm'](lstm_in,states)
  
          return lstm_out, hidden
  </code></pre>
  <p>Later on, we call upon this model in the training step:</p>
  <pre><code>def train(*args):
        
      for epoch in range(epochs):
          ....
          *init_zero_states
          ...
          out, states = model(input, states)
          ...
      return model
  </code></pre>
  <p>Let's just say, that I have 3 sentences as input:</p>
  <pre><code>sents = [[The, sun, is, shiny],
   [The, beach, was, very, windy],
   [Computer, broke, down, today]]
  </code></pre>
  <pre><code>model = train(LSTM, sents)
  </code></pre>
  <p>All words in all sentences gets converted to embeddings and loaded into the model.</p>
  <p>Now the question:</p>
  <ol>
  <li><p>Does the self.model['lstm'] iterate though all words from all articles and makes one output after every word? or every sentence?</p>
  </li>
  <li><p>How does the model make distinction between the 3 sentences, such as after getting &quot;The&quot;, &quot;sun&quot;, &quot;is&quot;, &quot;shiny&quot;, does something (such as the states) in the 'lstm' reset and begin anew?</p>
  </li>
  <li><p>The &quot;out&quot; in the train step after <code>out, states = model(input, states)</code> is the output after running all 3 sentences and hence the combined &quot;information&quot; from all 3 sentences?</p>
  </li>
  </ol>
  <p>Thanks!</p>