title: Using LSTM layers in combination with CNN
body: <p>I am trying to improve the accuracy of an EEG classifier. Currently I was only classifying with conv layers and fully connected ones. In the literature I found people using LSTM layers in their CNN model. I would like to try this out but I get following error.</p>
  
  <p>The data that I'm using is time series EEG data 64 channels x 325 samples (sampled at 500 Hz so 650 ms). X = (1923,63,325,1) and y = (1923,)</p>
  
  <pre><code>model = Sequential()
  
  model.add(TimeDistributed(Conv2D(64, (1, 3), input_shape=X.shape[1:])))
  model.add(Activation('relu'))
  
  model.add(TimeDistributed(Conv2D(64, (1, 3))))
  model.add(Activation('relu')) 
  model.add(MaxPooling2D(pool_size=(1, 2)))
  
  model.add(TimeDistributed(Conv2D(64, (1, 3))))
  model.add(Activation('relu'))
  model.add(MaxPooling2D(pool_size=(1, 2)))
  
  model.add(TimeDistributed(Flatten()))
  
  model.add(LSTM(128,return_sequences=True))
  
  model.add(LSTM(128,return_sequences=True))
  
  model.add(Dense(16))
  model.add(Activation('relu'))
  model.add(Dropout(0.3)) 
  
  model.add(Dense(2))
  model.add(Activation('softmax'))
  
  model.compile(loss='sparse_categorical_crossentropy',
                    optimizer='adam',
                    metrics=['accuracy'])
  return model
  </code></pre>
  
  <p>The error that I'm getting:</p>
  
  <pre><code>IndexError: list index out of range
  </code></pre>
  
  <p>I have seen some other questions about this online but most of them do not really apply on my application I think.</p>
  
  <p>Edit: error gets thrown</p>
  
  <pre><code>  File "&lt;ipython-input-8-0b3d7307ea53&gt;", line 1, in &lt;module&gt;
      runfile('D:/ AA TestPrograms/LALALAL/ModelV20.py', wdir='D:/ AA TestPrograms/LALALAL')
  
    File "d:\anaconda\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 668, in runfile
      execfile(filename, namespace)
  
    File "d:\anaconda\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
      exec(compile(f.read(), filename, 'exec'), namespace)
  
    File "D:/ AA TestPrograms/LALALAL/ModelV20.py", line 115, in &lt;module&gt;
      history = train_model(model, xtrain, ytrain, xval, yval)
  
    File "D:/ AA TestPrograms/LALALAL/ModelV20.py", line 94, in train_model
      shuffle=True
  
    File "d:\anaconda\lib\site-packages\keras\engine\training.py", line 950, in fit
      batch_size=batch_size)
  
    File "d:\anaconda\lib\site-packages\keras\engine\training.py", line 671, in _standardize_user_data
      self._set_inputs(x)
  
    File "d:\anaconda\lib\site-packages\keras\engine\training.py", line 577, in _set_inputs
      self.build(input_shape=(None,) + inputs.shape[1:])
  
    File "d:\anaconda\lib\site-packages\keras\engine\sequential.py", line 225, in build
      x = layer(x)
  
    File "d:\anaconda\lib\site-packages\keras\engine\base_layer.py", line 457, in __call__
      output = self.call(inputs, **kwargs)
  
    File "d:\anaconda\lib\site-packages\keras\layers\wrappers.py", line 248, in call
      y = self.layer.call(inputs, **kwargs)
  
    File "d:\anaconda\lib\site-packages\keras\layers\convolutional.py", line 168, in call
      dilation_rate=self.dilation_rate)
  
    File "d:\anaconda\lib\site-packages\keras\backend\tensorflow_backend.py", line 3565, in conv2d
      data_format=tf_data_format)
  
    File "d:\anaconda\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 779, in convolution
      data_format=data_format)
  
    File "d:\anaconda\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 828, in __init__
      input_channels_dim = input_shape[num_spatial_dims + 1]
  
    File "d:\anaconda\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 615, in __getitem__
      return self._dims[key]
  
  IndexError: list index out of range
  </code></pre>
  
  <p>I am a beginner when it comes to Python and Tensorflow but I think it is the last line where it checks the dimensions? If I look in tensor_shape.py I can't find an IndexError.</p>
  
  <p>Thanks for the help!</p>