Why imbalanced data-set will bias the prediction model towards the more common class?
Let us say you have a skewed class (which is linearly inseparable, otherwise the prediction is bound to be correct always) like this:



Image Credit: Learning from Imbalanced Classes

Now you can clearly see, if you move the boundary downwards you will be miss-classifying much more blue dots than red dots. And we know the cost function of a ML algorithm roughly tries to reduce the miss-classification (by converting it into a real valued loss optimisation problem, instead of discrete mis-classification reduction).

Think of it as democracy, each individual point tells the classifier what to do. The classifier will always listen to majority points. In this case the classifier will be miss-classifying much more blue points rather than red points if moved downwards. So the bias is set to high, so the line moves upwards.

Mathematically speaking: A classifier will always try to minimise its loss function, which roughly corresponds to making correct classifications (making correct classifications is discrete, optimising loss is not). So, it will position the boundary as such to make minimum number of mistakes disregarding class type. In general if a class has more points in the data-set, a classification boundary will tend to to make more miss-classifications of that class compared to class with less points. Hence optimising the class with higher points becomes more of a priority for th classifier, and it sets its bias accordingly.