title: What is timestep in LSTM layers from Keras? and How choose the value for this parameter?
body: <p>I have problems to understand the parameter "timestep" in LSTM layer. I have found some meanings, but I am very confuse now. Some mention that it is the amount of data per batch-size that enters the model during training. Others, on the other hand, say that it is the number of occurrences of a cell within the LSTM layer, while the states are being passed from one cell to another.</p>
  
  <p>The point is that I have the following form of the training data:</p>
  
  <pre><code>(sequences, number of frames per sequence, width, height, channel = 1)
  (2000, 5, 80, 80, 1)
  </code></pre>
  
  <p>My model must predict the following sequence of frames, in this case 5 future frames. The model consists of a variational autoencoder, first I use 3D convolutional layers to compress the sequences of 5 frames, then I resize the size of the outputs so that I can enter the LSTM layer, who only accepts (batch, timestep, features).</p>
  
  <pre><code>Model: "sequential"
  ____________________________________________________________________________________________________
  Layer (type)                                 Output Shape                            Param #        
  ====================================================================================================
  conv3d (Conv3D)                              (None, 2, 27, 27, 32)                   19392          
  ____________________________________________________________________________________________________
  batch_normalization (BatchNormalization)     (None, 2, 27, 27, 32)                   128            
  ____________________________________________________________________________________________________
  conv3d_1 (Conv3D)                            (None, 1, 14, 14, 32)                   2654240        
  ____________________________________________________________________________________________________
  batch_normalization_1 (BatchNormalization)   (None, 1, 14, 14, 32)                   128            
  ____________________________________________________________________________________________________
  conv3d_2 (Conv3D)                            (None, 1, 7, 7, 64)                     3211328        
  ____________________________________________________________________________________________________
  batch_normalization_2 (BatchNormalization)   (None, 1, 7, 7, 64)                     256            
  ____________________________________________________________________________________________________
  flatten (Flatten)                            (None, 3136)                            0              
  ____________________________________________________________________________________________________
  reshape (Reshape)                            (None, 4, 784)                          0              
  ____________________________________________________________________________________________________
  
  
  lstm (LSTM)                                  (None, 64)                              217344         
  ____________________________________________________________________________________________________
  repeat_vector (RepeatVector)                 (None, 4, 64)                           0              
  ____________________________________________________________________________________________________
  lstm_1 (LSTM)                                (None, 4, 64)                           33024          
  ____________________________________________________________________________________________________
  time_distributed (TimeDistributed)           (None, 4, 784)                          50960          
  ____________________________________________________________________________________________________
  reshape_1 (Reshape)                          (None, 1, 7, 7, 64)                     0              
  ____________________________________________________________________________________________________
  
  
  conv3d_transpose (Conv3DTranspose)           (None, 2, 14, 14, 64)                   6422592        
  ____________________________________________________________________________________________________
  batch_normalization_3 (BatchNormalization)   (None, 2, 14, 14, 64)                   256            
  ____________________________________________________________________________________________________
  conv3d_transpose_1 (Conv3DTranspose)         (None, 4, 28, 28, 32)                   5308448        
  ____________________________________________________________________________________________________
  batch_normalization_4 (BatchNormalization)   (None, 4, 28, 28, 32)                   128            
  ____________________________________________________________________________________________________
  conv3d_transpose_2 (Conv3DTranspose)         (None, 8, 84, 84, 1)                    19361          
  ____________________________________________________________________________________________________
  batch_normalization_5 (BatchNormalization)   (None, 8, 84, 84, 1)                    4              
  ____________________________________________________________________________________________________
  cropping3d (Cropping3D)                      (None, 8, 80, 80, 1)                    0              
  ____________________________________________________________________________________________________
  cropping3d_1 (Cropping3D)                    (None, 5, 80, 80, 1)                    0              
  ====================================================================================================
  </code></pre>
  
  <p>I have finally used the RESHAPE layer to get into the LSTM layer, with shape (batch, 4, 784). In other words, I have called timestpe = 4. I think it should be 5, or not necessarily should be equal to the number of frames I want to predict.</p>
  
  <p>What is the true meaning of timestep in this case? Do I need to order the values ​​of my layers? I really appreciate your support.</p>
  
  <p>On the other hand, I am thinking of applying convolutional layers to each frame, and no longer to the entire 5-frame sequence, but frame by frame and then connect the outputs of the convolutional layers to LSTM layers, finally connect the output states of the LSTM layers of each frame, respecting the order of the frames, in this case I consider using timestpe = 1.</p>