Selecting most important features for multilinear regression
Be careful choosing features based on correlation! Yes it is true that features that are correlated with the response variable may be good predictors, however if the features are correlated with each other then you are introducing multicollinearity into your model, which is bad.
If you want to avoid this you should choose features which are correlated with your response variable but not with each other. For example...
y = a + b + c
Assume:
y is correlated with all three (a, b, c)
a is correlated with b
c is not correlated with a and b
You should only use one of either a or b and c to predict y.
With regard to choosing which model is best, you should use a combination of metrics. As you increase the number of features your R^2 will tend to increase regardless of model performance. AIC and BIC can be used only to compare similar models and the &quot;best&quot; model according to AIC/BIC will be the one with the lowest score relative to the other models.
In my opinion RMSE is the best indicator of a good model, but you should also evaluate R^2, AIC, and BIC.
If you are unsure of which features to use I suggest you try stepwise regression to evaluate many models quickly and (maybe) find the best one.