title: Why do my models shift between 100% accuracy and 60% accuracy?
body: <p>I am trying to classify data as walking or running (and eventually other activities) from x,y, and z accelerometer and gyroscope data on the thigh and shank (6 features) using a 1D convolution neural network and sweeping over different parameters. </p>
  
  <p>When I train and evaluate the models on my data, I sometimes get 100% accuracy and other times get 60% accuracy (there are some combinations of parameters that result in ~99%). The models don't look like they're overfitting from the training v validation loss curves, but I find it odd that I sometimes get a 100.000% and then others so low.</p>
  
  <p>To see if it is always the case I train and evaluate each model 15 times and take the mean and standard deviation. Most of the parameter combinations exhibit this behavior, with some not.</p>
  
  <p>For example (the value at the end is the accuracy against unseen data): </p>
  
  <pre class="lang-py prettyprint-override"><code>&gt;Standardize=False Filter=16 Kernel=3 Batch=32: #1: 59.701
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #2: 100.000
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #3: 100.000
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #4: 99.975
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #5: 100.000
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #6: 40.299
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #7: 100.000
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #8: 59.701
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #9: 59.701
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #10: 100.000
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #11: 59.701
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #12: 100.000
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #13: 59.701
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #14: 59.701
  &gt;Standardize=False Filter=16 Kernel=3 Batch=32: #15: 99.975
  </code></pre>
  
  <p>This is the model I am using:</p>
  
  <pre class="lang-py prettyprint-override"><code>    model = models.Sequential()
      model.add(Conv1D(filters=filt, kernel_size=kernel, activation='relu',
                       input_shape=(n_timesteps, n_features)))
      model.add(Conv1D(filters=filt, kernel_size=kernel, activation='relu'))
      model.add(layers.Dropout(0.5))
      model.add(MaxPooling1D(pool_size=2))
      model.add(layers.Flatten())
      model.add(layers.Dense(100, activation='relu'))
      model.add(layers.Dense(n_activities, activation='softmax'))
      model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
  
      # fit network
      history = model.fit(X_train, y_train, validation_data=(X_val, y_val),
                          epochs=epochs, batch_size=batch_size, verbose=verbose)
  
      # evaluate model
      _, accuracy = model.evaluate(X_val, y_val, batch_size=batch_size, verbose=0)
  </code></pre>
  
  <p>I sweep over filter size (16, 32), kernel size (3, 5), and batch size (16,32) and I also check standardized and non standardized data.</p>
  
  <p>I am also training on about 10,000+ windows of data</p>
  
  <p>Does this mean that my model is wrong / not working correctly? If so, is there any fix to this?</p>