title: How to structure panel data for LSTM and Keras?
body: <p>I am trying to figure out how to structure my dataset and build the X and y such that it will work with Keras' Stacked LSTM for sequence classification.</p>
  
  <p>I have panel data where I am trying to predict classifications. I am not entirely sure how to understand timesteps or how to properly craft data's shape given my panel data.</p>
  
  <pre><code># Libraries
  from keras.models import Sequential
  from keras.layers import LSTM, Dense
  import numpy as np
  import pandas as pd
  
  # Here is an example of my data
  df = pd.read_csv('https://raw.githubusercontent.com/rocketfish88/democ/master/sample2.csv')
  df
  </code></pre>
  
  <pre><code># Contains a handful of features, a target, year, and id of the observation
     id        year  x1 x2  x3  y
  0   A       2015   1   1   1  1
  1   A       2016   2   2   2  1
  2   A       2017   3   3   3  2
  3   A       2018   4   4   4  2
  4   B       2015   1   1   1  3
  5   B       2016   2   2   2  2
  6   B       2017   3   3   3  1
  7   B       2018   4   4   4  1
  8   C       2015   1   1   1  2
  9   C       2016   2   2   2  2
  10  C       2017   3   3   3  3
  11  C       2018   4   4   4  2
  </code></pre>
  
  <p>Keras.io presents the following with example:</p>
  
  <pre><code>data_dim = 16
  timesteps = 8
  num_classes = 10
  
  # expected input data shape: (batch_size, timesteps, data_dim)
  model = Sequential()
  model.add(LSTM(32, return_sequences=True,
                 input_shape=(timesteps, data_dim)))  # returns a sequence of vectors of dimension 32
  model.add(LSTM(32, return_sequences=True))  # returns a sequence of vectors of dimension 32
  model.add(LSTM(32))  # return a single vector of dimension 32
  model.add(Dense(10, activation='softmax'))
  
  model.compile(loss='categorical_crossentropy',
                optimizer='rmsprop',
                metrics=['accuracy'])
  
  # Generate dummy training data
  x_train = np.random.random((1000, timesteps, data_dim))
  y_train = np.random.random((1000, num_classes))
  
  # Generate dummy validation data
  x_val = np.random.random((100, timesteps, data_dim))
  y_val = np.random.random((100, num_classes))
  
  model.fit(x_train, y_train,
            batch_size=64, epochs=5,
            validation_data=(x_val, y_val))
  </code></pre>
  
  <p>I am fairly lost as to how to take my dataset and transform it into the proper shape of (size, timesteps, dimensions)</p>
  
  <p>I appreciate any help!</p>