title: Accuracy stays constant while training a keras model. What may be the reason?
body: <p>I have been trying to train a simple skip gram word2vec model using keras.
  After training for some time, i noticed the accuracy is not changing at all</p>
  
  <p>Things ive tried:
  1. Ive tried changing the optimizers to adam, rmsprop.
  2. For sgd, ive tried changing the learning in steps from 1.0 to 0.000001
  3. Ive tried changing the batch size.</p>
  
  <p>Still no luck.</p>
  
  <p>Here's some of the code.</p>
  
  <pre><code>def make_model(self,vocab_size=256,vec_dim=100):
      model=Sequential()
      model.add(Dense(vec_dim,activation="sigmoid",input_dim=vocab_size))
      model.add(Dense(vocab_size,activation="sigmoid"))
      sgd=SGD(lr=1.0)
      model.compile(loss="categorical_crossentropy",optimizer=sgd,metrics=["accuracy"])
      return model
  
  def _get_callbacks(self):
      earlystop=EarlyStopping(monitor="val_loss",min_delta=0.0001,patience=10,verbose=2)
      checkpoint=ModelCheckpoint("checkpt.hdf5",period=10,verbose=2)
      reducelr=ReduceLROnPlateau(monitor="val_loss",factor=0.1,patience=5,verbose=2)
      return [earlystop,checkpoint,reducelr]
  
  def train(self,model,X,y):
      model.fit(X,y,nb_epoch=1000,callbacks=self._get_callbacks(),validation_split=0.1,verbose=2,batch_size=300)
      model.save("model.hdf5")
  </code></pre>
  
  <p>X is a one hot vector of len 256 for every word</p>
  
  <p>y in a one hot vector of len 256 representing the skip word in the context of X.</p>
  
  <p>so for instance if the sequence is [2,6,5,7,9]
  X will be
  [5,5,5,5,7,7,7...]</p>
  
  <p>y will be
  [2,6,7,9,6,5,9...]
  and so on for every word in the sequence.</p>
  
  <p>This is what happens when i try to train:</p>
  
  <pre><code>9s - loss: 4.5012 - acc: 0.1794 - val_loss: 4.5873 - val_acc: 0.1327
  Epoch 2/1000
  
  9s - loss: 4.2679 - acc: 0.1801 - val_loss: 4.8339 - val_acc: 0.1327
  Epoch 3/1000
  
  9s - loss: 4.2363 - acc: 0.1801 - val_loss: 4.7040 - val_acc: 0.1327
  Epoch 4/1000
  
  9s - loss: 4.2102 - acc: 0.1801 - val_loss: 4.6947 - val_acc: 0.1327
  Epoch 5/1000
  
  9s - loss: 4.1882 - acc: 0.1801 - val_loss: 4.6625 - val_acc: 0.1327
  Epoch 6/1000
  
  9s - loss: 4.1777 - acc: 0.1801 - val_loss: 4.6303 - val_acc: 0.1327
  </code></pre>
  
  <p>I ve waited for a about 50 epochs and the acc still does not change.</p>
  
  <p>Any idea what i am doing wrong? Ive faced this problem everytime ive used keras even when training other models like language modelling using RNNs text generation using LSTMs.</p>