Adding a bias makes Q-learning algorithm ineffective
Having a bias term with the one-hot encoding prevents each state’s Q values from being independent

Any ideas why this is the case?

The usual point of using linear regression or other function approximation in Q-learning is to generalise, and thus prevent the Q values from being independent deliberately. So this is not a general statement about Q-learning and linear function approximation. In general yes you can add bias terms - also, in general when using linear regression or neural networks to estimate action values, then you should have bias terms.
Having said that, I would expect the bias should make little difference here, since the one hot coding of state makes it redundant - you can already express any mapping of state to Q value of each possible action using just the weights. Although you are using what looks like a linear regression model, essentially this is just the tabular form of Q learning.
Q learning with function approximation can be unstable. The combination of off-policy, bootstrap updates and function approximation is known as the &quot;deadly triad&quot;, and you need to add tricks like experience replay to keep it stable. Although in theory the linear regression should learn a correct bias, probably adding bias puts the algorithm into that unstable zone.