title: More efficient way of filtering users on past behaviour
body: <p>I have two datasets: one recommendation output and other one activity data set which gives information regarding what activity or what content a user has browsed in past 2 months.</p>
  
  <p>Sample recommendation data</p>
  
  <pre><code>id  content
    1   100
    1   101
    2   102
    2   103
    3   103
    4   105 
  </code></pre>
  
  <p>Sample useractivity data</p>
  
  <pre><code>id  content
  1              34
  5              23
  2              102
  2              103
  3              103
  6              100
  </code></pre>
  
  <p>My objective is to remove those content from recommendation dataset for a user , which he has seen in past 2 months as depicted in the activity dataset. Now I take the common ids between these two datasets and for common ids I extract subsets of the two data for each id and add the end, add the uncommon ids to the output.</p>
  
  <p>Present solution works, but its horribly inefficient in terms of large datasets. Actual datasets have 30,000+ user ids. If someone can help in some other method in term of list comprehension or any other method, it would be appreciated</p>
  
  <pre><code>common_muids = list(set(useractivity_ids).intersection(reco_ids))
  
  
  final_rec1 = reco[reco.masteruserid.isin(common_muids)]
  final_rec2 = reco[~(reco.masteruserid.isin(common_muids))]
  
  d=DataFrame()
  
  for i in common_muids:
      final_rec_reduced=final_rec1[final_rec1.id==i]
      useractivity_reduced=useractivity[useractivity.id==i]
      useractivity_reduced_tbids=useractivity_reduced.tbid.unique().tolist()
      final_rec_reduced=final_rec_reduced[~(       final_rec_reduced.tbid.isin(useractivity_reduced_tbids))]
      d=d.append(final_rec_reduced)
  </code></pre>
  
  <p>At the end:</p>
  
  <pre><code>d=d.append(final_rec2)
  </code></pre>
  
  <p>Final output is:</p>
  
  <pre><code>id  tbid
  1   100
  1   101
  4   105
  </code></pre>