Will Q-learning converge to the optimal state-action function when the reward periodically changes?
No, it will not converge in the general case (maybe it might in extremely convenient special cases, not sure, didn't think hard enough about that...). 

Practically everything in Reinforcement Learning theory (including convergence proofs) relies on the Markov property; the assumption that the current state $s_t$ includes all relevant information, that the history leading up to $s_t$ is no longer relevant. In your case, this property is violated; it is important to remember whether or not we visited $s$ more recently than $s'$. 

I suppose if you "enhance" your states such that they include that piece of information, then it should converge again. This means that you'd essentially double your state-space. For every state that you have in your "normal" state space, you'd have to add a separate copy that would be used in cases where $s$ was visited more recently than $s'$.