Validation accuracy for neural network
You can't assume the loss wouldn't drop with additional cycles, but it's generally safe to assume there are diminishing returns. This is why stopping rules are defined in terms of both number of iterations and change in the loss function. You want to stop when the gains become vanishingly small. At that point, increasing your accuracy from, say, 90.1% to 90.2% probably have little practical benefit, and performance on a hold-out sample probably won't differ much. But, that is an empirical question you can actually test. Run a model to near completion then run another with more stringent stopping criteria and compare on a hold-out.