<h4 dir="auto">Code Sample</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import pandas as pd

df = pd.DataFrame({
    'a': [1, 1, 2, 3],
    'b': [None, 1, 2, 3],
    'c': [1,2,3,4],
})

grouped = df.groupby(('a', 'b'))
print('Number of groups:', len(grouped))
# Number of groups: 4

num_iterations = 0
for _ in grouped:
    num_iterations += 1

print('Number of groups iterated over:', num_iterations) 
# Number of groups iterated over: 3"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>

<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>({
    <span class="pl-s">'a'</span>: [<span class="pl-c1">1</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>],
    <span class="pl-s">'b'</span>: [<span class="pl-c1">None</span>, <span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>],
    <span class="pl-s">'c'</span>: [<span class="pl-c1">1</span>,<span class="pl-c1">2</span>,<span class="pl-c1">3</span>,<span class="pl-c1">4</span>],
})

<span class="pl-s1">grouped</span> <span class="pl-c1">=</span> <span class="pl-s1">df</span>.<span class="pl-en">groupby</span>((<span class="pl-s">'a'</span>, <span class="pl-s">'b'</span>))
<span class="pl-en">print</span>(<span class="pl-s">'Number of groups:'</span>, <span class="pl-en">len</span>(<span class="pl-s1">grouped</span>))
<span class="pl-c"># Number of groups: 4</span>

<span class="pl-s1">num_iterations</span> <span class="pl-c1">=</span> <span class="pl-c1">0</span>
<span class="pl-k">for</span> <span class="pl-s1">_</span> <span class="pl-c1">in</span> <span class="pl-s1">grouped</span>:
    <span class="pl-s1">num_iterations</span> <span class="pl-c1">+=</span> <span class="pl-c1">1</span>

<span class="pl-en">print</span>(<span class="pl-s">'Number of groups iterated over:'</span>, <span class="pl-s1">num_iterations</span>) 
<span class="pl-c"># Number of groups iterated over: 3</span></pre></div>
<h4 dir="auto">Problem description</h4>
<p dir="auto">Presently, when a <code class="notranslate">GroupBy</code> object is iterated over, any group where one of the columns grouped by is <code class="notranslate">None</code> is skipped.</p>
<p dir="auto">This is a problem because when iterating over groups, we expect to iterate over all groups.</p>
<p dir="auto">It is also a problem because it is not sensible to say the length of an iterable is <code class="notranslate">x</code> when iterating over it only performs some <code class="notranslate">y &lt; x</code> number of iterations.</p>
<h4 dir="auto">Expected Output</h4>
<p dir="auto">I'd expect iterating over a <code class="notranslate">GroupBy</code> object to iterate over all groups, regardless of the value in the key columns.</p>
<h4 dir="auto">Output of <code class="notranslate">pd.show_versions()</code></h4>
<details>
INSTALLED VERSIONS
<hr>
<p dir="auto">commit: None<br>
python: 3.5.2.final.0<br>
python-bits: 64<br>
OS: Linux<br>
OS-release: 2.6.32-504.12.2.el6.x86_64<br>
machine: x86_64<br>
processor: x86_64<br>
byteorder: little<br>
LC_ALL: None<br>
LANG: en_US.UTF-8</p>
<p dir="auto">pandas: 0.18.1<br>
nose: 1.3.7<br>
pip: 9.0.1<br>
setuptools: 23.0.0<br>
Cython: 0.24<br>
numpy: 1.11.1<br>
scipy: 0.17.1<br>
statsmodels: 0.6.1<br>
xarray: None<br>
IPython: 5.1.0<br>
sphinx: 1.4.1<br>
patsy: 0.4.1<br>
dateutil: 2.5.3<br>
pytz: 2016.4<br>
blosc: None<br>
bottleneck: 1.1.0<br>
tables: 3.3.0<br>
numexpr: 2.6.0<br>
matplotlib: 1.5.1<br>
openpyxl: 2.3.2<br>
xlrd: 1.0.0<br>
xlwt: 1.1.2<br>
xlsxwriter: 0.9.2<br>
lxml: 3.6.0<br>
bs4: 4.4.1<br>
html5lib: None<br>
httplib2: None<br>
apiclient: None<br>
sqlalchemy: 1.0.13<br>
pymysql: None<br>
psycopg2: 2.6.2 (dt dec pq3 ext)<br>
jinja2: 2.8<br>
boto: 2.40.0<br>
pandas_datareader: None</p>
</details>
<p dir="auto">ENH: maybe for now just provide a warning if dropping the nan rows when pivotting...</p>
<p dir="auto">rom ml</p>
<p dir="auto"><a href="http://stackoverflow.com/questions/16860172/python-pandas-pivot-table-silently-drops-indices-with-nans" rel="nofollow">http://stackoverflow.com/questions/16860172/python-pandas-pivot-table-silently-drops-indices-with-nans</a></p>
<p dir="auto">This is effectivly trying to groupby on a NaN, currently not allowed</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [13]: a = [['a', 'b', 12, 12, 12], ['a', nan, 12.3, 233., 12], ['b', 'a', 123.23, 123, 1], ['a', 'b', 1, 1, 1.]]

In [14]: df = DataFrame(a, columns=['a', 'b', 'c', 'd', 'e'])

In [15]: df.groupby(['a','b']).sum()
Out[15]: 
          c    d   e
a b                 
a b   13.00   13  13
b a  123.23  123   1"><pre class="notranslate"><code class="notranslate">In [13]: a = [['a', 'b', 12, 12, 12], ['a', nan, 12.3, 233., 12], ['b', 'a', 123.23, 123, 1], ['a', 'b', 1, 1, 1.]]

In [14]: df = DataFrame(a, columns=['a', 'b', 'c', 'd', 'e'])

In [15]: df.groupby(['a','b']).sum()
Out[15]: 
          c    d   e
a b                 
a b   13.00   13  13
b a  123.23  123   1
</code></pre></div>
<p dir="auto">Workaround to fill the index with a dummy, pivot, and replace</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
    In [31]: df2 = df.copy()

    In [32]: df2['dummy'] = np.nan

    In [33]: df2['b'] = df2['b'].fillna('dummy')

    In [34]: df2
    Out[34]: 
       a      b       c    d   e  dummy
    0  a      b   12.00   12  12    NaN
    1  a  dummy   12.30  233  12    NaN
    2  b      a  123.23  123   1    NaN
    3  a      b    1.00    1   1    NaN

    In [35]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum)
    Out[35]: 
       a      b       c    d   e
    0  a      b   13.00   13  13
    1  a  dummy   12.30  233  12
    2  b      a  123.23  123   1

    In [36]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum).replace('dummy',np.nan)
    Out[36]: 
       a    b       c    d   e
    0  a    b   13.00   13  13
    1  a  NaN   12.30  233  12
    2  b    a  123.23  123   1
"><pre class="notranslate"><code class="notranslate">
    In [31]: df2 = df.copy()

    In [32]: df2['dummy'] = np.nan

    In [33]: df2['b'] = df2['b'].fillna('dummy')

    In [34]: df2
    Out[34]: 
       a      b       c    d   e  dummy
    0  a      b   12.00   12  12    NaN
    1  a  dummy   12.30  233  12    NaN
    2  b      a  123.23  123   1    NaN
    3  a      b    1.00    1   1    NaN

    In [35]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum)
    Out[35]: 
       a      b       c    d   e
    0  a      b   13.00   13  13
    1  a  dummy   12.30  233  12
    2  b      a  123.23  123   1

    In [36]: df2.pivot_table(rows=['a', 'b'], values=['c', 'd', 'e'], aggfunc=sum).replace('dummy',np.nan)
    Out[36]: 
       a    b       c    d   e
    0  a    b   13.00   13  13
    1  a  NaN   12.30  233  12
    2  b    a  123.23  123   1

</code></pre></div>