title: PySpark AND EXISTS equivalent to sql query
body: <p>I have following sql query and I need same result in pySpark:</p>
  <pre><code>Select * 
  From FIRSTKEY o 
  Where o.INCRNR=123456
  And exists ( 
  select 1 
  from SECONDKEY b 
  where o.ID = b.ID 
  and o.IPDAT = b.IPDAT            
  and b.INCRNR=123456 
  and b.TYPE = '02'  
  )
  </code></pre>
  <p>Especially the <code>and exists</code> condition is one that I cannot resolve. I saw <a href="https://stackoverflow.com/questions/62499029/pyspark-query-and-sql-pyspark-query">here</a>, that it can be done by <code>join</code> or <code>LEFT SEMI JOIN</code> but as its still not clear to me how to use it I would be grateful to see it on example above.  Thanks for any advice!</p>
  <p>EDIT: I need it with build in functions as because of other transformations of data. Something like:</p>
  <pre class="lang-py prettyprint-override"><code>def incrementalTransformFunction(output, fsInput, fsInput2):
      input_df = fsInput.dataframe('added')
      input2_df = fsInput2.dataframe('added')
      # output
      output.write_dataframe(input_df.drop(*drop_list).filter(input_df.incrnr==123456)
  .andexists((b.id == o.id) &amp; (o.ipdat == b.ipdat) &amp; (b.incrnr == 123456) &amp; (b.type == '02')))
  
  </code></pre>