title: multiple conditions for filter in spark data frames
body: <p>I have a data frame with four fields. one of the field name is Status and i am trying to use a OR condition in .filter for a dataframe . I tried below queries but no luck.</p>
  
  <pre><code>df2 = df1.filter(("Status=2") || ("Status =3"))
  
  df2 = df1.filter("Status=2" || "Status =3")
  </code></pre>
  
  <p>Has anyone used this before. I have seen a similar question on stack overflow <a href="https://stackoverflow.com/questions/33747834/sparksql-filtering-selecting-with-where-clause-with-multiple-conditions">here</a> . They have used below code for using OR condition. But that code is for pyspark.</p>
  
  <pre><code>from pyspark.sql.functions import col 
  
  numeric_filtered = df.where(
  (col('LOW')    != 'null') | 
  (col('NORMAL') != 'null') |
  (col('HIGH')   != 'null'))
  numeric_filtered.show()
  </code></pre>