title: Spark - How to avoid duplicate columns after join?
body: <p>Extending upon use case given here:
  <a href="https://stackoverflow.com/questions/35258506/how-to-avoid-duplicate-columns-after-join">How to avoid duplicate columns after join?</a></p>
  
  <p>I have two dataframes with the 100s of columns. Following are some samples with join columns:</p>
  
  <pre><code>df1.columns
  //  Array(ts, id, X1, X2, ...)
  </code></pre>
  
  <p>and    </p>
  
  <pre><code>df2.columns
  //  Array(ts, id, X1, Y2, ...)
  </code></pre>
  
  <p>After I do:</p>
  
  <pre><code>val df_combined = df1.join(df2, df1.X1===df2.X1 and df1.X2==df2.Y2)
  </code></pre>
  
  <p>I end up with the following columns: <code>Array(ts, id, X1, X2, ts, id, X1, Y2)</code>. <code>X1</code> is duplicated.</p>
  
  <p>I can't use <code>join(right: Dataset[_], usingColumns: Seq[String])</code> api as to use this api all columns must be there in both dataframe which is not the case here (<code>X2</code> and <code>Y2</code>). Only option I see is to rename a column and drop column later or to alias dataframe and drop column later from 2nd dataframe. 
  Isn't there a simple api to achieve this? E.g. automatically drop one of the join column in case of equality join.</p>