Different Initial Q-Values in Q-Learning
In the long-run, tabular Q-learning converges toward the optimal regardless of initialization.
However, the speed of convergence may be affected, similarly to an n-armed bandit setting : http://incompleteideas.net/book/first/ebook/node21.html
For more on initialization in Q learning, I recommend &quot;Potential-based shaping and Q-value initialization are equivalent&quot; by Eric Wiewiora.