title: Check if value from one dataframe column exists in another dataframe column using Spark Scala
body: <p>I have 2 dataframes df1 and df2,</p>
  
  <p><code>df1</code> has column Name with values like a,b,c etc
  <code>df2</code> has column Id   with values like a,b</p>
  
  <p>If Name column in <code>df1</code> has a match in Id column in <code>df2</code>, then we need to have match status as 0. If there is no match then we need to have match status as 1.
  I know that I can put <code>df2</code> ID column in a collection using collect and then check if Name column in df1 has matching entry.</p>
  
  <pre><code>val df1 = Seq(“Rey”, “John”).toDF(“Name”)
  val df2 = Seq(“Rey”).toDF(“Id”)
  
  val collect = df2.select("Id").map(r =&gt; r.getString(0)).collect.toList 
  </code></pre>
  
  <p>something like,</p>
  
  <pre><code>    val df3 = 
      df1.withColumn("match_sts",when(df1("Name").isin(collect).then(0).else(1)))
  
  Expected output
  + — — + — -+
  |Name|match_sts|
  + — — + — -+
  | Rey| 0  |
  |John| 1  |
  + — — + — -+
  </code></pre>
  
  <p>But I don't want to use collect here. Is there any alternate approach available.</p>