<p dir="auto">Hi,<br>
I encountered a 'problem' when my program tried to groupby a dataframe with an empty (ie full of nan) column. Here a code sample with pandas 0.19.2 :</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [4]: a = pd.DataFrame({'a': [2,2,3,4,5], 
                          'b': [5,5,6,6,7], 
                          'c': [np.nan]*5, 
                          'sum': [1]*5})

In [5]: a
Out[5]: 
   a  b   c  sum
0  2  5 NaN    1
1  2  5 NaN    1
2  3  6 NaN    1
3  4  6 NaN    1
4  5  7 NaN    1

In [6]: a.groupby(['a','b','c']).agg({'sum':'sum'})
Out[6]: 
Empty DataFrame
Columns: [sum]
Index: []

In [7]: a.groupby(['a','b']).agg({'sum':'sum'})
Out[7]: 
     sum
a b     
2 5    2
3 6    1
4 6    1
5 7    1"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">4</span>]: <span class="pl-s1">a</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>({<span class="pl-s">'a'</span>: [<span class="pl-c1">2</span>,<span class="pl-c1">2</span>,<span class="pl-c1">3</span>,<span class="pl-c1">4</span>,<span class="pl-c1">5</span>], 
                          <span class="pl-s">'b'</span>: [<span class="pl-c1">5</span>,<span class="pl-c1">5</span>,<span class="pl-c1">6</span>,<span class="pl-c1">6</span>,<span class="pl-c1">7</span>], 
                          <span class="pl-s">'c'</span>: [<span class="pl-s1">np</span>.<span class="pl-s1">nan</span>]<span class="pl-c1">*</span><span class="pl-c1">5</span>, 
                          <span class="pl-s">'sum'</span>: [<span class="pl-c1">1</span>]<span class="pl-c1">*</span><span class="pl-c1">5</span>})

<span class="pl-v">In</span> [<span class="pl-c1">5</span>]: <span class="pl-s1">a</span>
<span class="pl-v">Out</span>[<span class="pl-c1">5</span>]: 
   <span class="pl-s1">a</span>  <span class="pl-s1">b</span>   <span class="pl-s1">c</span>  <span class="pl-s1">sum</span>
<span class="pl-c1">0</span>  <span class="pl-c1">2</span>  <span class="pl-c1">5</span> <span class="pl-v">NaN</span>    <span class="pl-c1">1</span>
<span class="pl-c1">1</span>  <span class="pl-c1">2</span>  <span class="pl-c1">5</span> <span class="pl-v">NaN</span>    <span class="pl-c1">1</span>
<span class="pl-c1">2</span>  <span class="pl-c1">3</span>  <span class="pl-c1">6</span> <span class="pl-v">NaN</span>    <span class="pl-c1">1</span>
<span class="pl-c1">3</span>  <span class="pl-c1">4</span>  <span class="pl-c1">6</span> <span class="pl-v">NaN</span>    <span class="pl-c1">1</span>
<span class="pl-c1">4</span>  <span class="pl-c1">5</span>  <span class="pl-c1">7</span> <span class="pl-v">NaN</span>    <span class="pl-c1">1</span>

<span class="pl-v">In</span> [<span class="pl-c1">6</span>]: <span class="pl-s1">a</span>.<span class="pl-en">groupby</span>([<span class="pl-s">'a'</span>,<span class="pl-s">'b'</span>,<span class="pl-s">'c'</span>]).<span class="pl-en">agg</span>({<span class="pl-s">'sum'</span>:<span class="pl-s">'sum'</span>})
<span class="pl-v">Out</span>[<span class="pl-c1">6</span>]: 
<span class="pl-v">Empty</span> <span class="pl-v">DataFrame</span>
<span class="pl-v">Columns</span>: [<span class="pl-s1">sum</span>]
<span class="pl-v">Index</span>: []

<span class="pl-v">In</span> [<span class="pl-c1">7</span>]: <span class="pl-s1">a</span>.<span class="pl-en">groupby</span>([<span class="pl-s">'a'</span>,<span class="pl-s">'b'</span>]).<span class="pl-en">agg</span>({<span class="pl-s">'sum'</span>:<span class="pl-s">'sum'</span>})
<span class="pl-v">Out</span>[<span class="pl-c1">7</span>]: 
     <span class="pl-s1">sum</span>
<span class="pl-s1">a</span> <span class="pl-s1">b</span>     
<span class="pl-c1">2</span> <span class="pl-c1">5</span>    <span class="pl-c1">2</span>
<span class="pl-c1">3</span> <span class="pl-c1">6</span>    <span class="pl-c1">1</span>
<span class="pl-c1">4</span> <span class="pl-c1">6</span>    <span class="pl-c1">1</span>
<span class="pl-c1">5</span> <span class="pl-c1">7</span>    <span class="pl-c1">1</span></pre></div>
<p dir="auto">I think we could throw a warning (or maybe an error) when this case is encountered in addition to return an empty dataframe.</p>
<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.1.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-75-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
<p dir="auto">pandas: 0.19.2<br>
nose: 1.3.7<br>
pip: 9.0.1<br>
setuptools: 27.2.0<br>
Cython: 0.25.2<br>
numpy: 1.11.3<br>
scipy: 0.18.1<br>
statsmodels: 0.6.1<br>
xarray: None<br>
IPython: 5.1.0<br>
sphinx: 1.5.1<br>
patsy: 0.4.1<br>
dateutil: 2.6.0<br>
pytz: 2016.10<br>
blosc: None<br>
bottleneck: 1.2.0<br>
tables: 3.3.0<br>
numexpr: 2.6.1<br>
matplotlib: 2.0.0<br>
openpyxl: 2.4.1<br>
xlrd: 1.0.0<br>
xlwt: 1.2.0<br>
xlsxwriter: 0.9.6<br>
lxml: 3.7.2<br>
bs4: 4.5.3<br>
html5lib: None<br>
httplib2: None<br>
apiclient: None<br>
sqlalchemy: 1.1.5<br>
pymysql: 0.7.9.None<br>
psycopg2: None<br>
jinja2: 2.9.4<br>
boto: 2.45.0<br>
pandas_datareader: None</p>
</details>
<p dir="auto">ENH: maybe for now just provide a warning if dropping the nan rows when pivotting...</p>
<p dir="auto">rom ml</p>
<p dir="auto"><a href="http://stackoverflow.com/questions/16860172/python-pandas-pivot-table-silently-drops-indices-with-nans" rel="nofollow">http://stackoverflow.com/questions/16860172/python-pandas-pivot-table-silently-drops-indices-with-nans</a></p>
<p dir="auto">This is effectivly trying to groupby on a NaN, currently not allowed</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [13]: a = [['a', 'b', 12, 12, 12], ['a', nan, 12.3, 233., 12], ['b', 'a', 123.23, 123, 1], ['a', 'b', 1, 1, 1.]]

In [14]: df = DataFrame(a, columns=['a', 'b', 'c', 'd', 'e'])

In [15]: df.groupby(['a','b']).sum()
Out[15]: 
          c    d   e
a b                 
a b   13.00   13  13
b a  123.23  123   1"><pre class="notranslate"><code class="notranslate">In [13]: a = [['a', 'b', 12, 12, 12], ['a', nan, 12.3, 233., 12], ['b', 'a', 123.23, 123, 1], ['a', 'b', 1, 1, 1.]]

In [14]: df = DataFrame(a, columns=['a', 'b', 'c', 'd', 'e'])

In [15]: df.groupby(['a','b']).sum()
Out[15]: 
          c    d   e
a b                 
a b   13.00   13  13
b a  123.23  123   1
</code></pre></div>
<p dir="auto">Workaround to fill the index with a dummy, pivot, and replace</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
    In [31]: df2 = df.copy()

    In [32]: df2['dummy'] = np.nan

    In [33]: df2['b'] = df2['b'].fillna('dummy')

    In [34]: df2
    Out[34]: 
       a      b       c    d   e  dummy
    0  a      b   12.00   12  12    NaN
    1  a  dummy   12.30  233  12    NaN
    2  b      a  123.23  123   1    NaN
    3  a      b    1.00    1   1    NaN

    In [35]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum)
    Out[35]: 
       a      b       c    d   e
    0  a      b   13.00   13  13
    1  a  dummy   12.30  233  12
    2  b      a  123.23  123   1

    In [36]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum).replace('dummy',np.nan)
    Out[36]: 
       a    b       c    d   e
    0  a    b   13.00   13  13
    1  a  NaN   12.30  233  12
    2  b    a  123.23  123   1
"><pre class="notranslate"><code class="notranslate">
    In [31]: df2 = df.copy()

    In [32]: df2['dummy'] = np.nan

    In [33]: df2['b'] = df2['b'].fillna('dummy')

    In [34]: df2
    Out[34]: 
       a      b       c    d   e  dummy
    0  a      b   12.00   12  12    NaN
    1  a  dummy   12.30  233  12    NaN
    2  b      a  123.23  123   1    NaN
    3  a      b    1.00    1   1    NaN

    In [35]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum)
    Out[35]: 
       a      b       c    d   e
    0  a      b   13.00   13  13
    1  a  dummy   12.30  233  12
    2  b      a  123.23  123   1

    In [36]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum).replace('dummy',np.nan)
    Out[36]: 
       a    b       c    d   e
    0  a    b   13.00   13  13
    1  a  NaN   12.30  233  12
    2  b    a  123.23  123   1

</code></pre></div>