title: Removing nulls from Pyspark Dataframe in individual columns
body: <p>I have a pyspark dataframe like this:</p>
  
  <p><code>+--------------------+--------------------+
  |                name|               value|
  +--------------------+--------------------+
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                null|                null|
  |                  id|                null|
  |                name|                null|
  |                 age|                null|
  |                food|                null|
  |                null|                   1|
  |                null|                 Joe|
  |                null|                  47|
  |                null|               pizza|
  +--------------------+--------------------+</code></p>
  
  <p>I want to remove the null values from each individual columns so the non-null data lines up. </p>
  
  <p>The desired output is:</p>
  
  <p><code>+--------------------+--------------------+
  |                name|               value|
  +--------------------+--------------------+
  |                  id|                   1|
  |                name|                 Joe|
  |                 age|                  47|
  |                food|               pizza|
  +--------------------+--------------------+</code></p>
  
  <p>I have tried removing nulls doing something like <code>df.dropna(how='any'/'all')</code> but and by separating out the columns and removing the nulls, but then it becomes difficult to join them back together.</p>