title: pyspark; check if an element is in collect_list
body: <p>I am working on a dataframe <code>df</code>, for instance the following dataframe:</p>
  
  <pre><code>df.show()
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code>+----+------+
  |keys|values|
  +----+------+
  |  aa| apple|
  |  bb|orange|
  |  bb|  desk|
  |  bb|orange|
  |  bb|  desk|
  |  aa|   pen|
  |  bb|pencil|
  |  aa| chair|
  +----+------+
  </code></pre>
  
  <p>I use <code>collect_set</code> to aggregate and get a <em>set of objects with duplicate elements eliminated</em> (or <code>collect_list</code> to get list of objects). </p>
  
  <pre><code>df_new = df.groupby('keys').agg(collect_set(df.values).alias('collectedSet_values'))
  </code></pre>
  
  <p>The resulting dataframe is then as follows:</p>
  
  <pre><code>df_new.show()
  </code></pre>
  
  <p>Output:</p>
  
  <pre><code>+----+----------------------+
  |keys|collectedSet_values   |
  +----+----------------------+
  |bb  |[orange, pencil, desk]|
  |aa  |[apple, pen, chair]   |
  +----+----------------------+
  </code></pre>
  
  <p>I am struggling to find a way to see if a specific keyword (like 'chair') is in the resulting set of objects (in column <code>collectedSet_values</code>). I do not want to go with <code>udf</code> solution. </p>
  
  <p>Please comment your solutions/ideas.</p>
  
  <p>Kind Regards.</p>