title: Spark 1.6 drop columns based on value
body: <pre><code>val max_col = df.select(df.columns.map(c =&gt; max(c)): _*)
  </code></pre>
  
  <p>gives me a 1 row dataframe with multiple columns. Many of these columns have value 0, i.e. there are no other values than 0 in these columns (all are positive numbers).</p>
  
  <p>How do I select only the columns with max value > 0? I.e. how do I drop all columns (not rows, there are plenty of details on how to drop rows) where max greater than 0?</p>
  
  <pre><code>a b c d
  
  1 7 0 1
  </code></pre>
  
  <p>Would like to have </p>
  
  <pre><code>a b d
  
  1 7 1
  </code></pre>
  
  <p>With </p>
  
  <pre><code>val exprs = max_col.columns.map(c =&gt; when(col(c) !== 0, col(c)).otherwise("null").as(c))
  val df_new = max_col.select(exprs:_*)
  </code></pre>
  
  <p>I can turn above to </p>
  
  <pre><code>a b c d
  
  1 7 null 1
  </code></pre>
  
  <p>but no success in finding na.drop for columns.</p>
  
  <p>Seems like a 'newbie' question, but didn't find a nice way, yet (I surfed the web for hours, realising (again) that Google certainly have problems with their service). </p>
  
  <p>Any help would be much appreciated. Very.</p>