Keras Early Stopping: Monitor 'loss' or 'val_loss'?
Validation loss. Overall training loss should keep decreasing so monitoring it isn't as meaningful. If your issue is noise in the validation loss, increase patience.