Why transforming the data to a high-dimensional feature space in which classes are linearly separable leads to overfitting?
@ffriend has a good post about it, but generally speaking, if you transform to a high dimensional feature space and train from there, the learning algorithm is 'forced' to take into account the higher-space features, even though they might have nothing to do with the original data, and offer no predictive qualities. 

This means that you are not going to be properly generalizing a learning rule when training. 

Take an intuitive example: Suppose you wanted to predict weight from height. You have all this data, corresponding to people's weights and heights. Let us say that very generally, they follow a linear relationship. That is, you can describe weight (W) and height (H) as:

$$
W = mH - b
$$

, where $m$ is the slope of your linear equation, and $b$ is the y-intercept, or in this case, the W-intercept. 

Let us say that you are a seasoned biologist, and that you know that the relationship is linear. Your data looks like a scatter plot trending upwards. If you keep the data in the 2-dimensional space, you will fit a line through it. It might not hit all the points, but thats ok - you know that the relationship is linear, and you want a good approximation anyway. 

Now lets say that you took this 2-dimensional data and transformed it into higher dimensional space. So instead of only $H$, you also add 5 more dimensions, $H^2$, $H^3$, $H^4$, $H^5$, and $\sqrt{H^2 + H^7}$. 

Now you go and find co-efficients of the polynomial to fit this data. That is, you want to find co-efficients $c_i$ for the this polynomial that 'best fits' the data:

$$
W = c_1H + c_2H^2 + c_3H^3 + c_4H^4 + c_5H^5 + c_6\sqrt{H^2+H^7}
$$

If you do that, what kind of line would you get? You would get one that looked a lot like @ffriend 's far right plot. You have overfit the data, because you 'forced' your learning algorithm to take into account higher order polynomials that have nothing to do with anything. Biologically speaking, weight just depends on height linearly. It doesnt depend on $\sqrt{H^2 + H^7}$ or any higher order nonsense. 

This is why if you transform the data to higher order dimensions blindly, you run a very bad risk of overfitting, and not generalizing.