Behaviour of Learning Algorithms on Random Data
They will of course still learn some best decision boundary. We know it will be meaningless, but there will still be better and best coefficients for the algorithm to learn when fitting to this particular instance of data from this random process. It may produce better than 50% accuracy on the data set, but of course this is purely due to overfitting whatever the data happens to be. It will not predict future outcomes with more than 50% accuracy.