Loss Function - Decreasing a lot at beginning of the epoch
Have you already plotted the training and validation loss over multiple epochs? What you would expect is something that looks like an exponential decay.

Chances are your models learns the mean of the target variable in the first few batches, which reduces the MSE-loss already considerably and only later learns the subtle differences in your data. You can try to use different batch sizes as well.

Are you generally happy with the performance on your model? Does it perform well on the test/validation set? If this is the case there might be no reason to be worried about.