In the linear regression, when we have a categorical explanatory variable with $n$ levels, we usually remove one level and call it a baseline level and fit the model on the remaining levels. And the final intercept is the intercept plus the coefficient of baseline level. Now my questions are:Does it matter which level I choose to remove? I am working on a dataset to predict house price. When I use linear model with intercept on the testset for prediction, I get negative values for some houses which is not correct, as house price cannot be negative. But when I fit a regression without an intercept all the prediction values are positive as expected. I guess it has to do something with the baseline level I chose. Do I have to remove one level from all my categorical variables in the dataset before fit a random forest, KNN, Ridge, Lasso,... too?!
Handling categorical variables in linear regression and random forest
random forest;linear regression;categorical data