Linear vs Non linear regression (Basic Beginner) So my doubt is basically in Linear regression, We try to fit a straight line or a curve for a given training set. Now, I believe whenever the features (independent variable) increases, parameters also increases. Hence computing these parameters is computationally expensive. So, I guess that's the reason we move to Non linear!?  Is my understanding right?

And my next doubt is, in overfitting for linear regression, we say that the model memorizes. What I understand is that the parameters are adjusted in such a way that it can only predict for the given input and will give bad results for output
 And the reason is because we haven't fitted a generalized line or curve. But we are plotting a line or curve that passes through all output values. Is my understanding correct?
Whether you use a linear model or a non-linear model depends completely on whether or not the data is linearly separable, not the number of features. If you have a dataset that is linearly separable, i.e a linear curve can determine the dependent variable, you would use linear regression irrespective of the number of features. Since real-world data is rarely linearly separable and linear regression does not provide accurate results on such data, non-linear regression is used. 

And to answer your second question, yes you are correct. Overfitting occurs when your model has a very high accuracy for training data but a considerably low accuracy for validation/test data. It happens because the parameters are adjusted completely based on your training set, so there is not a generalised curve. Hence on newer data, the model would perform badly.