title: Finding all pairs of users that have a certain amount of common values
body: <p>I am new to spark and I am trying to find specific information about a couple of lists of data that I have converted into two separate DataFrames. </p>
  
  <p>The two DataFrames are:</p>
  
  <pre><code>Users:                item_Details:
  user_id | item_id     item_id | item_name
  -----------------     ----------------------
    1     | 123           123   |  phone
    2     | 223           223   |  game
    3     | 423           423   |  foo
    2     | 1223          1223  |  bar
    1     | 3213          3213  | foobar
  </code></pre>
  
  <p>I need to find all pairs of users that have more than 50 common items and sorted on the number of items. There can be no duplicates meaning there should only be one set of userId 1 and userId 2.</p>
  
  <p>The result needs to look like this:</p>
  
  <pre><code>user_id1 | user_id2 | count_of_items | list_of_items
  -------------------------------------------------------------
      1    |     2    |       51       |  phone,foo,bar,foobar
  </code></pre>