NGBoost and overfit - which model is used?
No, I'm afraid you won't get the best model if you don't ask for it specifically.
But don't despair - just set the fit parameter of early_stopping_rounds to a number - and it will stop after this number of rounds in which the validation loss got worse.
The number of rounds that would work best for you should be set experimentally (i.e., just fiddle around fit till you get best results, but don't forget hyperparameter overfit...)
PS.
The remarks in the code claim that this number of rounds is consecutive, but looking a the code it does not look like the count is reset each time validation loss improves (the counter val_loss_list is not reset).
You could change this manually be editing ngboost.py.