title: What is the correct LSTM input shape for a multivariate time-series?
body: <p>I have a multivariate timeseries with 5 features (5 independent timeseries). With these 5 features, I want to predict a target <code>y</code> which is also a timeseries. I have some ~50,000 observations in the data.</p>
  <pre><code>data  # pandas dataframe
  &gt;&gt;&gt;
              x1        x2        x3        x4        x5        y
  time
  18:20:00    0.462     -0.248   -0.873    0.892      0.012     0.938
  18:21:00    0.621     -0.399   -0.772    0.891      0.008     0.922
  18:22:00    0.726     -0.401   -0.771    0.899      0.009     0.910
  ... # 50,000 rows
  </code></pre>
  <p>I have seen multiple different sources, blogs and papers that all use slightly different setups and architectures when constructing an LSTM [<a href="https://stackoverflow.com/questions/38714959/understanding-keras-lstms?rq=1">1</a>][<a href="https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/" rel="nofollow noreferrer">2</a>][<a href="https://machinelearningmastery.com/how-to-develop-lstm-models-for-time-series-forecasting/" rel="nofollow noreferrer">3</a>][<a href="https://github.com/DeepWolf90/DLSTM2/tree/master/case_study_1(Chinese%20oil)/DLSTM" rel="nofollow noreferrer">4</a>].</p>
  <p>I understand that the input has to be of shape <code>[n_samples, n_timesteps, n_features]</code>. I believe that two of the three dimensions are already answered from my data; <code>n_samples=50000</code> and <code>n_features=5</code> since I have 50k samples and 5 features. The dimension <code>n_timesteps</code> is the confusing one.</p>
  <p>If I reshape my data so that each &quot;sample&quot; input into the LSTM has overlapping time observations as such</p>
  <pre><code>import numpy as np
  
  def reshape_data(data, n_steps):
      out = np.empty((data.shape[0] - n_steps + 1, n_steps, data.shape[1]))
      for i in range(data.shape[0] - n_steps + 1):
          out[i] = data[i: i + n_steps, :]
      return out
  
  n_step = 10
  reshaped_data = (data[[&quot;x1&quot;, &quot;x2&quot;, &quot;x3&quot;, &quot;x4&quot;, &quot;x5&quot;]].values, n_step)
  reshaped_data.shape
  &gt;&gt;&gt; (49991, 10, 5)
  
  # target variable also needs to be truncated
  # to match the shape and time index of `reshaped_data`
  target = data[&quot;y&quot;].values
  target = target[n_step - 1:]
  target.shape
  &gt;&gt;&gt; (49991,)
  </code></pre>
  <p>The data is now essentially a &quot;3d tensor&quot;, or a matrix of matrices where each matrix contains 10 observations in time. Matrix one contains observations from time <code>t0</code> to time <code>t9</code> inclusive, then matrix two contains observations from time <code>t1</code> to <code>t10</code> inclusive and so on, so we can see each sample has 9 overlapping observations with the last. Each of these matrices is a sample given to the LSTM - there are <code>n_samples</code> of these matrices (49991 in our example). <strong>Now <code>[n_samples, n_timesteps, n_features] = (49991, 10, 5)</code>.</strong></p>
  <p>I can now input the above data into an LSTM</p>
  <pre><code>model = Sequential()
  model.add(LSTM(50, return_sequences=True, input_shape=(reshaped_data.shape[1], reshaped_data.shape[2])))
  model.add(Dropout(0.4))
  model.add(LSTM(50, return_sequences=False))
  model.add(Dropout(0.4))
  model.add(Dense(1))
  model.compile(optimizer='adam', loss='mse')
  model.fit(reshaped_data, target, epochs=50, batch_size=32, verbose=1, shuffle=False)
  </code></pre>
  <p><strong>Alternatively</strong>, I could just take the original data and shape it as the following, where there are no overlapping observations in time for each sample</p>
  <pre><code>reshaped_data = data[[&quot;x1&quot;, &quot;x2&quot;, &quot;x3&quot;, &quot;x4&quot;, &quot;x5&quot;]].values
  reshaped_data = reshaped_data.reshape((reshaped_data.shape[0], 1, reshaped_data[1]))
  reshaped_data.shape
  (50000, 1, 5)
  </code></pre>
  <p>Now I can also give this data to an LSTM. In this case, <code>[n_samples, n_timesteps, n_features] = (50000, 1, 5)</code>, so each sample only has one observation in time.</p>
  <p>Here is my confusion</p>
  <ol>
  <li>From the two methods of reshaping data above, which one is correct? Which one should lead to better results (higher accuracy)? What is the difference when training an LSTM with either reshaping method?</li>
  <li>How does <code>batch_size</code> effect training with either data rehsaping method above?</li>
  </ol>