What should be the criteria to select features using correlation factors between features?
There are several questions here, but let me try to answer the overall issue.  How do I know which features to drop or add to make the best model?


There is no rule that can be used to say “always drop all but one feature if X correlation exists”.  I usually start with > +/- 0.8 but you should test on a per dataset basis.
feature engineering, creating new features or eliminating features, is the art side of analytics.  Take into account what you know about the dataset, what you can learn from data analysis and experimenting.   Often all of the experiments don’t get captured in data kernels (especially those that don’t have positive impacts) so it just looks like magic to people reviewing the final/posted solution.