Training CNN on a huge data set
It reminds me of this question, the training loss is decreasing faster than the validation loss. I understand there is some overfitting, as the model is learning some patterns that are only in the training set, but the model is still learning some patterns that are more general, as the validation loss is decreasing as well. To me it would be more of an issue if the validation loss increased, but it is not the case.
Edit
Usually neural networks are trained with all the data, training by using mini-batch gradient descent already does what you mention in your approach without the need of storing the model in memory. So, I would train with as much data as possible, to have a model with the least possible variance. If you are not feeding the data using generators and the whole dataset doesn't fit into memory, I recommend to use them, or train with a model which is as big as possible given your memory limitations.