title: Remove strongly correlated columns from DataFrame
body: <p>I  have a DataFrame like this </p>
  
  <pre><code>dict_ = {'Date':['2018-01-01','2018-01-02','2018-01-03','2018-01-04','2018-01-05'],'Col1':[1,2,3,4,5],'Col2':[1.1,1.2,1.3,1.4,1.5],'Col3':[0.33,0.98,1.54,0.01,0.99]}
  df = pd.DataFrame(dict_, columns=dict_.keys())
  </code></pre>
  
  <p>I then calculate the pearson correlation between the columns and filter out columns that are correlated above my threshold of 0.95</p>
  
  <pre><code>def trimm_correlated(df_in, threshold):
      df_corr = df_in.corr(method='pearson', min_periods=1)
      df_not_correlated = ~(df_corr.mask(np.eye(len(df_corr), dtype=bool)).abs() &gt; threshold).any()
      un_corr_idx = df_not_correlated.loc[df_not_correlated[df_not_correlated.index] == True].index
      df_out = df_in[un_corr_idx]
      return df_out
  </code></pre>
  
  <p>which yields</p>
  
  <pre><code>uncorrelated_factors = trimm_correlated(df, 0.95)
  print uncorrelated_factors
  
      Col3
  0   0.33
  1   0.98
  2   1.54
  3   0.01
  4   0.99
  </code></pre>
  
  <p>So far I am happy with the result, but I would like to keep one column from each correlated pair, so in the above example I would like to include Col1 or Col2. To get s.th. like this</p>
  
  <pre><code>    Col1   Col3
  0    1     0.33
  1    2     0.98
  2    3     1.54
  3    4     0.01
  4    5     0.99
  </code></pre>
  
  <p>Also on a side note, is there any further evaluation I can do to determine which of the correlated columns to keep?</p>
  
  <p>thanks </p>