If loss reduction means model improvement, why doesn't accuracy increase? Problem Statement

I've built a classifier to classify a dataset consisting of n samples and four classes of data. To this end, I've used pre-trained VGG-19, pre-trained Alexnet and even LeNet (with cross-entropy loss). However, I just changed the softmax layer's architecture and placed just four neurons for that (because my dataset includes just four classes). Since the dataset classes have a striking resemblance to each other, this classifier was unable to classify them and I was forced to use other methods.
During the training section, after some epochs, loss decreased from approximately 7 to approximately 1.2, but there were no changes in accuracy and it was frozen on 25% (random precision). In the best epochs, the accuracy just reached near 27% but it was completely unstable.

Question

How is it justifiable? If loss reduction means model improvement, why doesn't accuracy increase? How is it possible to the loss decreases near 6 points (approximately from 7 to 1) but nothing happens to accuracy at all?
Loss reduction means model improvement, it does not in the wrong setup, wher random choise produces least loss. So it is some critical setup error. What classes do you have?
I got also thet recently experimenting with an encoder with too narrow coding layer - it just EQUILIZES the output with average values cause this state has minimum loss.