title: Spark dataFrame taking too long to display after updating its columns
body: <p>I have a dataFrame of approx. 4 million rows and 35 columns as input.</p>
  
  <p>All I do to this dataFrame is the following steps:</p>
  
  <ul>
  <li>For a list of given columns, I calculate a sum for a given list of group features and joined it as new column to my input dataFrame</li>
  <li>I drop each new column sum right after I joined it to the dataFrame.</li>
  </ul>
  
  <p>Therefore we end up with the same dataFrame as we started from (in theory).</p>
  
  <p>However, I noticed that if my list of given columns gets too big (from more than 6 columns), the output dataFrame becomes impossible to manipulate. Even a simple display takes 10 minutes.</p>
  
  <p>Here is an example of my code (df is my input dataFrame):</p>
  
  <pre><code>  for c in list_columns:
      df = df.join(df.groupby(list_group_features).agg(sum(c).alias('sum_' + c)), list_group_features)
      df = df.drop('sum_' + c)
  </code></pre>