Why does Q-learning converge to the optimal policy, even if the agent acts sub-optimally?
Q-learning is an off-policy learning algorithm. We are following the behaviour policy, $b$, which is $\epsilon-$greedy. This behaviour policy need not be an optimal policy rather it is a more explorable policy. But we are learning the target policy, $\pi$, which is argmax of state action value $(Q(s,a))$. This target policy is by definition optimal policy.
From the $\epsilon$-greedy policy improvement theorem we can show that for any $\epsilon$-greedy policy (I think you are referring to this as a non-optimal policy) we are still making progress towards the optimal policy and when $\pi^{'}$ = $\pi$ that is our optimal policy (Rich Sutton's book Chapter-5). Here $\pi^{'}$ is the new policy and $\pi$ is the previous policy.

Think of this diagram, where we are selecting action based on $\epsilon$-greedy policy but still making progress towards the optimal policy $\pi_*$.