Behaviour of Learning Algorithms on Random Data Suppose we collect data for 100,000 tosses of a fair coin and record "Heads" or "Tails" as the value for the attribute outcome and also record the time, temprature and other irrelevant attributes.

We know that the outcome of each toss is random so there should be no way of predicting future unlabeled data instances.

My question is how do learning algorithms (support vector machines, for example) behave when we apply them on random data such as this?
They will of course still learn some best decision boundary. We know it will be meaningless, but there will still be better and best coefficients for the algorithm to learn when fitting to this particular instance of data from this random process. It may produce better than 50% accuracy on the data set, but of course this is purely due to overfitting whatever the data happens to be. It will not predict future outcomes with more than 50% accuracy.