title: pyspark how do we check if a column value is contained in a list
body: <p>I'm trying to figure out if there is a function that would check if a column of a spark DataFrame contains any of the values in a list: </p>
  
  <pre class="lang-python prettyprint-override"><code># define a dataframe
  rdd = sc.parallelize([(0,100), (0,1), (0,2), (1,2), (1,10), (1,20), (3,18), (3,18), (3,18)])
  df = sqlContext.createDataFrame(rdd, ["id", "score"])
  
  # define a list of scores
  l = [1]
  
  # filter out records by scores by list l
  records = df.filter(~df.score.contains(l))
  
  # expected: (0,100), (0,1), (1,10), (3,18)
  </code></pre>
  
  <p>I get an issue running this code : </p>
  
  <blockquote>
    <p><code>java.lang.RuntimeException: Unsupported literal type class java.util.ArrayList [1]</code></p>
  </blockquote>
  
  <p>Is there a way to do this or do we have to loop through the list to pass contains? </p>