title: groupby keeping all previous fields
body: <p>I have duplicate entries that I need to combine. All fields are the same except for two fields called <code>id1</code> and <code>id2</code> -- these are list fields and I want to combine their entries. Here is how I would do that for only the <code>id1</code> and <code>id2</code> fields:</p>
  <pre><code>summary_df = df.groupby(['path_md5']).agg(
      id1 =('id1', lambda x: str(sorted({id for ids in x.dropna() for id in ids}))),
      id2 =('id2', lambda x: str(sorted({id for ids in x.dropna() for id in ids}))),
  )
  </code></pre>
  <p>Yet, I don't want to add in 60 additional fields that say <code>first</code> so that I can grab their value. Is there a better way to do this? Here is an example of input/output I want:</p>
  <pre><code>id1     id2     path_md5       other_fields (could be 50 fields -- all the same)...
  [1,2]   [3]     abc            ...
  [7]     [9]     abc            ...
  [17]    [11]    xyz            ...
  </code></pre>
  <p>And the result should be:</p>
  <pre><code>id1        id2     path_md5       other_fields...
  [1,2,7]   [3,9]     abc            ...
  [17]      [11]      xyz            ...
  </code></pre>
  <p>What would be the best way to do this?</p>
  <hr />
  <p>I tried doing the following:</p>
  <pre><code># Dedupe path, combining id1, id2
  agg_fields = [col_name for col_name in df.columns if col_name not in ('id1', 'id2')]
  raw_df = raw_df.groupby(agg_fields).agg(...).reset_index()
  </code></pre>
  <p>But it was giving me zero results (maybe since a lot of the values are null?</p>