My accuracy changes throughout every epoc but the val_acc at the end of each epoc stays the same
Your model complexity is high enough to learn features and give you results better than 20%. Although I would suggest to increase your learning rate to 1e-3 or 1e-4.

The accuracy in epoch refers to accuracy for the batch its training on so it might fluctuate. But is val accuracy and loss is same that means the model isn't learning distiguishable features. It could mean the problem is hard.