title: Combine 2 data frames with different columns in spark
body: <p>I have 2 dataframes:</p>
  
  <p>df1 :</p>
  
  <pre><code>Id    purchase_count   purchase_sim
  12       100               1500
  13       1020              1300
  14       1010              1100
  20       1090              1400
  21       1300              1600
  </code></pre>
  
  <p>df2:</p>
  
  <pre><code>Id     click_count      click_sim
  12       1030              2500
  13       1020              1300
  24       1010              1100
  30       1090              1400
  31       1300              1600
  </code></pre>
  
  <p>I need to get the combined data frame with results as :</p>
  
  <pre><code>Id     click_count      click_sim     purchase_count   purchase_sim
  12       1030              2500            100               1500
  13       1020              1300            1020              1300
  14       null              null            1010              1100
  24       1010              1100            null              null
  30       1090              1400            null              null
  31       1300              1600            null              null
  20       null              null            1090              1400
  21       null              null            1300              1600                                     
  </code></pre>
  
  <p>I can't use union because of different column names. Can some one suggest me a better way to do this ?</p>