Improve a regression model and feature selection
I think the next option you have to take is to add more features. You have a huge amount of training examples, which is good, but the number of features is very low. Adding more features is one of the most used methods to improve performance in machine learning. 

Furthermore, it would be good to try to understand how your features affect your model. Imagine you have a linear model like y = theta1*feat1 + theta2*feat2 + theta3*feat3. If theta3 is near to 0, then feat3 is not affecting the model.