If $\alpha$ decreases over time, why is Q-learning guaranteed to converge?
Why is this a convergence criterion?

It is because $R$ and $S'$ are stochastic. A large learning rate applied when these values have variance would not converge to mean, but would wander around typically within some value proportional to $\alpha\sigma$ of the true value, where $\sigma$ is the standard deviation of the term $R + \gamma\text{max}_aQ(S',a)$. If you reduce $\alpha$ towards zero, then this expected error will also reduce to zero.
For deterministic environments, it should be possible to prove convergence with large $\alpha$.

In the special case of static policy, tabular learning and $\alpha = \frac{1}{N(s,a)}$ where $N(s,a)$ is number of visits to state $s$, action $a$, then the expected error for each Q value is the MSE from basic stats i.e. $\frac{\sigma_{TD}}{\sqrt{N(s,a)}}$