Conceptual questions on MLP and Perceptrons
When the data is linearly inseparable, we use MLP. Here what is meant by "data"--is it the response or the input feature that is linearly inseparable?


This means that a linear function of the input features is unable to separate the response.

To answer your question a bit more directly: Given only a linear function of the inputs, the response is the thing that's inseparable.


  If it is linearly inseparable then does it mean that the mapping function from input to output will always be non-linear? Hence, we prefer MLP or the latest new models such as deep learning?


Yes.  If the mapping from input to output were linear, then the output would necessarily be linearly separable by the input.


  Linear regression fails in the case of linearly inseparable data or can linear regression work for inseparable data but if the function mapping is nonlinear then it fails? 


Linear regression will never be able to perfectly separate linearly inseparable data.  Consider the following example, where the input features are x1 and x2, and the output is the color:



It doesn't matter how you draw a line in the 2D space - you'll never be able to separate the colors.  The same idea applies in higher dimensions.

I hope that helps!