title: How to filter a dataframe by multiple columns?
body: <p>I have a problem as below:</p>
  
  <p>I have two dataframes</p>
  
  <p>Dataframe DF1:</p>
  
  <pre><code>ID, Name, age
  1   name1  18
  2   name2  20
  </code></pre>
  
  <p>DataFrame DF2:</p>
  
  <pre><code>ID, Name, age
  1   name1 18
  3   name3 19
  </code></pre>
  
  <p>I am attempting to filter DF2 to exclude records contained in DF1 by ID and Name so that I can get new DF2 like</p>
  
  <pre><code>ID, Name, age
  3   name3  19
  </code></pre>
  
  <p>and then union these two dataframes to get final result:</p>
  
  <pre><code>ID, Name, age
  1   name1  18
  2   name2  20
  3   name3  19
  </code></pre>
  
  <p>To do this in T-SQL, I can write a statement like</p>
  
  <pre class="lang-sql prettyprint-override"><code>INSERT INTO DF1 
  SELECT ID, Name, age FROM DF2 WHERE NOT EXISTS
  (SELECT 1 FROM DF1 WHERE DF1.ID = DF2.ID AND DF1.Name = DF2.Name)
  </code></pre>
  
  <p>But I find that "insert" is not supported in dataframe in sparkSQL.
  So my questions are: </p>
  
  <p>How can I filter a dataframe based on multiple column?</p>
  
  <p>How can I union two dataframes together?
  I am appreciate for any solution.</p>