Why not continue next epoch with best weights
A new epoch starts with the same weights as the previous epoch but with different mini-batch of data. Model performance will have small oscillations across mini-batches for that reason.

Always starting each epoch with the best overall weights might result in staying in a local optimum. Gradient descent is already prone to staying in a local optimum, at the expense of finding a better optimum. Encouraging continued exploration of weights is one of the goals of training.

There is always a chance that both training and validation loss will do down. It is okay that training performance gets worse, there is no need for monotonic improvement in training. 

At the end of the training, you should select the weights that have the lowest validation loss.