How to determine convergence when using Q-learning?
There are two ways that I know of:
First, based on how many iterations, like you would do.
Second, based on the difference between current Q value and previous Q value (i.e., insignificant difference means termination).