Automatically Remove highly correlated features using Variance Inflation Factor? I want to be able to automatically remove highly correlated features. I am performing a classification problem using a set of 20-30 features and some may be correlated. 

Multiple features can be correlated at once too and I fear it may pose a problem in my Logit model significances &amp; coefficients of the features. After removing these features I plan to then also use this reduced feature set into an Xgboost model as well.


  Multicolinearity on the other hand is more troublesome to detect because it emerges when three or more variables, which are highly correlated, are included within a model. (Reference here)


Would it be correct to remove correlated variables this way?:

X=data[feature_set]
vif = pd.DataFrame()
vif['vif_factor'] = [variance_inflation_factor(X.values,i) for i in range(X.shape[1])]
vif['features'] = X.columns

vif.sort_values('vif_factor',axis=0,inplace=True, ascending=False)

features_to_remove = vif.loc[vif['vif_factor'] &gt; 10,'features'].values
features_to_remove = list(features_to_remove)
print(features_to_remove)


Output: 

vif_factor |  feature

21         |   age
9.7        |   income
7          |   gender ....and so on 


So in this case age would be removed as a feature from the model.
I might oversimplify this, but Pandas allows to drop correlated features based on a threshold. E.g. correlation >0.95



https://chrisalbon.com/machine_learning/feature_selection/drop_highly_correlated_features/