<p dir="auto">Hi,<br>
I updated to version 0.23.0 and all of a sudden the following code breaks:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import pandas as pd
df = pd.DataFrame(data={'date': list(pd.date_range('5.1.2018', '5.10.2018')),
                        'vals': list(range(10))})
df.groupby([df.date.dt.month, df.date.dt.day])['vals'].sum()

ValueError: Duplicated level name: &quot;date&quot;, assigned to level 1, is already used for level 0."><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>
<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>(<span class="pl-s1">data</span><span class="pl-c1">=</span>{<span class="pl-s">'date'</span>: <span class="pl-en">list</span>(<span class="pl-s1">pd</span>.<span class="pl-en">date_range</span>(<span class="pl-s">'5.1.2018'</span>, <span class="pl-s">'5.10.2018'</span>)),
                        <span class="pl-s">'vals'</span>: <span class="pl-en">list</span>(<span class="pl-en">range</span>(<span class="pl-c1">10</span>))})
<span class="pl-s1">df</span>.<span class="pl-en">groupby</span>([<span class="pl-s1">df</span>.<span class="pl-s1">date</span>.<span class="pl-s1">dt</span>.<span class="pl-s1">month</span>, <span class="pl-s1">df</span>.<span class="pl-s1">date</span>.<span class="pl-s1">dt</span>.<span class="pl-s1">day</span>])[<span class="pl-s">'vals'</span>].<span class="pl-en">sum</span>()

<span class="pl-v">ValueError</span>: <span class="pl-v">Duplicated</span> <span class="pl-s1">level</span> <span class="pl-s1">name</span>: <span class="pl-s">"date"</span>, <span class="pl-s1">assigned</span> <span class="pl-s1">to</span> <span class="pl-s1">level</span> <span class="pl-c1">1</span>, <span class="pl-c1">is</span> <span class="pl-s1">already</span> <span class="pl-s1">used</span> <span class="pl-k">for</span> <span class="pl-s1">level</span> <span class="pl-c1">0.</span></pre></div>
<h3 dir="auto">Expected output:</h3>
<p dir="auto">Using version 0.22.0 the same code yields the following:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="date  date
5     1       0
      2       1
      3       2
      4       3
      5       4
      6       5
      7       6
      8       7
      9       8
      10      9
Name: vals, dtype: int64"><pre class="notranslate"><span class="pl-s1">date</span>  <span class="pl-s1">date</span>
<span class="pl-c1">5</span>     <span class="pl-c1">1</span>       <span class="pl-c1">0</span>
      <span class="pl-c1">2</span>       <span class="pl-c1">1</span>
      <span class="pl-c1">3</span>       <span class="pl-c1">2</span>
      <span class="pl-c1">4</span>       <span class="pl-c1">3</span>
      <span class="pl-c1">5</span>       <span class="pl-c1">4</span>
      <span class="pl-c1">6</span>       <span class="pl-c1">5</span>
      <span class="pl-c1">7</span>       <span class="pl-c1">6</span>
      <span class="pl-c1">8</span>       <span class="pl-c1">7</span>
      <span class="pl-c1">9</span>       <span class="pl-c1">8</span>
      <span class="pl-c1">10</span>      <span class="pl-c1">9</span>
<span class="pl-v">Name</span>: <span class="pl-s1">vals</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span></pre></div>
<p dir="auto">It obviously contains duplicated level names. I get why this might be a problem, but as of version 0.23.0 it's not possible to specify the resulting level names.</p>
<p dir="auto">Opening a new issue so this isn't lost.</p>
<p dir="auto">In <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="283691748" data-permission-text="Title is private" data-url="https://github.com/pandas-dev/pandas/issues/18882" data-hovercard-type="pull_request" data-hovercard-url="/pandas-dev/pandas/pull/18882/hovercard" href="https://github.com/pandas-dev/pandas/pull/18882">#18882</a> banned duplicate names in a MultiIndex. I think this is a good change since allowing duplicates hit a lot of edge cases when you went to actually do something. I want to make sure we understand all the cases that actually produce duplicate names in the MI though, specifically groupby.apply.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [1]: import dask.dataframe as dd

In [2]: import pandas as pd

In [3]:     pdf = pd.DataFrame({'a': [1, 2, 3, 4, 5, 6, 7, 8, 9],
   ...:                         'b': [4, 5, 6, 3, 2, 1, 0, 0, 0]},
   ...:                        index=[0, 1, 3, 5, 6, 8, 9, 9, 9]).set_index(&quot;a&quot;)
   ...:
   ...:

In [4]: pdf.groupby(pdf.index).apply(lambda x: x.b)"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">1</span>]: <span class="pl-s1">import</span> <span class="pl-s1">dask</span>.<span class="pl-s1">dataframe</span> <span class="pl-k">as</span> <span class="pl-s1">dd</span>

<span class="pl-v">In</span> [<span class="pl-c1">2</span>]: <span class="pl-s1">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>

<span class="pl-v">In</span> [<span class="pl-c1">3</span>]:     <span class="pl-s1">pdf</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>({<span class="pl-s">'a'</span>: [<span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>, <span class="pl-c1">4</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>, <span class="pl-c1">7</span>, <span class="pl-c1">8</span>, <span class="pl-c1">9</span>],
   ...:                         <span class="pl-s">'b'</span>: [<span class="pl-c1">4</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>, <span class="pl-c1">3</span>, <span class="pl-c1">2</span>, <span class="pl-c1">1</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>]},
   ...:                        <span class="pl-s1">index</span><span class="pl-c1">=</span>[<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">3</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>, <span class="pl-c1">8</span>, <span class="pl-c1">9</span>, <span class="pl-c1">9</span>, <span class="pl-c1">9</span>]).<span class="pl-en">set_index</span>(<span class="pl-s">"a"</span>)
   ...:
   ...:

<span class="pl-v">In</span> [<span class="pl-c1">4</span>]: <span class="pl-s1">pdf</span>.<span class="pl-en">groupby</span>(<span class="pl-s1">pdf</span>.<span class="pl-s1">index</span>).<span class="pl-en">apply</span>(<span class="pl-k">lambda</span> <span class="pl-s1">x</span>: <span class="pl-s1">x</span>.<span class="pl-s1">b</span>)</pre></div>
<p dir="auto">Another, more realistic example: groupwise drop_duplicates:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [18]: df = pd.DataFrame({&quot;B&quot;: [0, 0, 0, 1, 1, 1, 2, 2, 2]}, index=pd.Index([0, 1, 1, 2, 2, 2, 0, 0, 1], name='a'))

In [19]: df
Out[19]:
   B
a
0  0
1  0
1  0
2  1
2  1
2  1
0  2
0  2
1  2

In [20]: df.groupby('a').apply(pd.DataFrame.drop_duplicates)
Out[20]:
     B
a a
0 0  0
  0  2
1 1  0
  1  2
2 2  1"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">18</span>]: <span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>({<span class="pl-s">"B"</span>: [<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">2</span>, <span class="pl-c1">2</span>]}, <span class="pl-s1">index</span><span class="pl-c1">=</span><span class="pl-s1">pd</span>.<span class="pl-v">Index</span>([<span class="pl-c1">0</span>, <span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">2</span>, <span class="pl-c1">2</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">1</span>], <span class="pl-s1">name</span><span class="pl-c1">=</span><span class="pl-s">'a'</span>))

<span class="pl-v">In</span> [<span class="pl-c1">19</span>]: <span class="pl-s1">df</span>
<span class="pl-v">Out</span>[<span class="pl-c1">19</span>]:
   <span class="pl-v">B</span>
<span class="pl-s1">a</span>
<span class="pl-c1">0</span>  <span class="pl-c1">0</span>
<span class="pl-c1">1</span>  <span class="pl-c1">0</span>
<span class="pl-c1">1</span>  <span class="pl-c1">0</span>
<span class="pl-c1">2</span>  <span class="pl-c1">1</span>
<span class="pl-c1">2</span>  <span class="pl-c1">1</span>
<span class="pl-c1">2</span>  <span class="pl-c1">1</span>
<span class="pl-c1">0</span>  <span class="pl-c1">2</span>
<span class="pl-c1">0</span>  <span class="pl-c1">2</span>
<span class="pl-c1">1</span>  <span class="pl-c1">2</span>

<span class="pl-v">In</span> [<span class="pl-c1">20</span>]: <span class="pl-s1">df</span>.<span class="pl-en">groupby</span>(<span class="pl-s">'a'</span>).<span class="pl-en">apply</span>(<span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>.<span class="pl-s1">drop_duplicates</span>)
<span class="pl-v">Out</span>[<span class="pl-c1">20</span>]:
     <span class="pl-v">B</span>
<span class="pl-s1">a</span> <span class="pl-s1">a</span>
<span class="pl-c1">0</span> <span class="pl-c1">0</span>  <span class="pl-c1">0</span>
  <span class="pl-c1">0</span>  <span class="pl-c1">2</span>
<span class="pl-c1">1</span> <span class="pl-c1">1</span>  <span class="pl-c1">0</span>
  <span class="pl-c1">1</span>  <span class="pl-c1">2</span>
<span class="pl-c1">2</span> <span class="pl-c1">2</span>  <span class="pl-c1">1</span></pre></div>
<p dir="auto">Is it possible to throw a warning on this for now, in case duplicate names are more common than we thought?</p>