What is the risk of removing a feature with low correlation...? I'm running a linear regression model as baseline for a specific estimation problem. Based on the resulting R-squared, regressor coefficients and their respective p-values, I can conclude that specific independent variables can be removed from the model.


What is the induced risk of removing these variables from the feature set?
Can other models—that are better at modelling nonlinear relationships—suffer from this decision?
How can I be sure that I am not loosing valuable nonlinear information without running "nonlinear" regressors?
Variable selection in linear regression is based on partial correlations, not zero order correlations. The partial correlation is the one resulting after a conditioning variable effect is removed from both X and Y. 

While a variable may be non highly correlated with Y, it may be highly correlated with an X variable in such a way that the X variable after being partialled out of the conditioning effect becomes highly significant. 

This effect is known as suppression among other names. 
HTH