Why does loss decrease but accuracy decreases too (Pytorch, LSTM)?
I have built a model with LSTM - Linear modules in Pytorch for a classification problem (10 classes). I am training the model and for each epoch I output the loss and accuracy in the training set. The ouput is as follows:

epoch: 0 start!
  Loss: 2.301875352859497
  Acc: 0.11388888888888889
  epoch: 1 start!
  Loss: 2.2759320735931396
  Acc: 0.29
  epoch: 2 start!
  Loss: 2.2510263919830322
  Acc: 0.4872222222222222
  epoch: 3 start!
  Loss: 2.225804567337036
  Acc: 0.6066666666666667
  epoch: 4 start!
  Loss: 2.199286699295044
  Acc: 0.6511111111111111
  epoch: 5 start!
  Loss: 2.1704766750335693
  Acc: 0.6855555555555556
  epoch: 6 start!
  Loss: 2.1381614208221436
  Acc: 0.7038888888888889
  epoch: 7 start!
  Loss: 2.1007182598114014
  Acc: 0.7194444444444444
  epoch: 8 start!
  Loss: 2.0557992458343506
  Acc: 0.7283333333333334
  epoch: 9 start!
  Loss: 1.9998993873596191
  Acc: 0.7427777777777778
  epoch: 10 start!
  Loss: 1.9277743101119995
  Acc: 0.7527777777777778
  epoch: 11 start!
  Loss: 1.8325848579406738
  Acc: 0.7483333333333333
  epoch: 12 start!
  Loss: 1.712520718574524
  Acc: 0.7077777777777777
  epoch: 13 start!
  Loss: 1.6056485176086426
  Acc: 0.6305555555555555
  epoch: 14 start!
  Loss: 1.5910680294036865
  Acc: 0.4938888888888889
  epoch: 15 start!
  Loss: 1.6259561777114868
  Acc: 0.41555555555555557
  epoch: 16 start!
  Loss: 1.892195224761963
  Acc: 0.3655555555555556
  epoch: 17 start!
  Loss: 1.4949012994766235
  Acc: 0.47944444444444445
  epoch: 18 start!
  Loss: 1.4332982301712036
  Acc: 0.48833333333333334  

For loss function I have used nn.CrossEntropyLoss and Adam Optimizer.
Although the loss is constantly decreasing, the accuracy increases until epoch 10 and then begins for some reason to decrease.

Why is this happening ?

Even if my model is overfitting, doesn't that mean that the accuracy should be high ?? (always speaking for accuracy and loss measured on the training set, not the validation set)

Decreasing loss does not mean improving accuracy always. 
I will try to address this for the cross-entropy loss. 

CE-loss= sum (-log p(y=i))

Note that loss will decrease if the probability of correct class increases and loss increases if the probability of correct class decreases. Now, when you compute average loss, you are averaging over all the samples, some of the probabilities may increase and some of them can decrease, making overall loss smaller but also accuracy drops.   
