What are the conditions for the convergence of SARSA to the optimal value function?
I have the conditions for convergence in these notes SARSA convergence by Nahum Shimkin.


The Robbins-Monro conditions above hold for $α_t$.
Every state-action pair is visited infinitely often
The policy is greedy with respect to the policy derived from $Q$ in the limit
The controlled Markov chain is communicating: every state can be reached from any other with positive probability (under some policy).
$\operatorname{Var}{R(s, a)} &lt; \infty$, where $R$ is the reward function