<h4 dir="auto">Code Sample, a copy-pastable example if possible</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="df = pd.DataFrame({'x': pd.Series([['a', 'b'], ['q']]),
                   'y': pd.Series([['z'], ['q', 't']])})
df.index = pd.MultiIndex.from_tuples([('i0', 'j0'), ('i1', 'j1')])

df.apply(lambda row: [el for el in row['x'] if el in row['y']], axis=1)"><pre class="notranslate"><span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>({<span class="pl-s">'x'</span>: <span class="pl-s1">pd</span>.<span class="pl-v">Series</span>([[<span class="pl-s">'a'</span>, <span class="pl-s">'b'</span>], [<span class="pl-s">'q'</span>]]),
                   <span class="pl-s">'y'</span>: <span class="pl-s1">pd</span>.<span class="pl-v">Series</span>([[<span class="pl-s">'z'</span>], [<span class="pl-s">'q'</span>, <span class="pl-s">'t'</span>]])})
<span class="pl-s1">df</span>.<span class="pl-s1">index</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">MultiIndex</span>.<span class="pl-en">from_tuples</span>([(<span class="pl-s">'i0'</span>, <span class="pl-s">'j0'</span>), (<span class="pl-s">'i1'</span>, <span class="pl-s">'j1'</span>)])

<span class="pl-s1">df</span>.<span class="pl-en">apply</span>(<span class="pl-k">lambda</span> <span class="pl-s1">row</span>: [<span class="pl-s1">el</span> <span class="pl-k">for</span> <span class="pl-s1">el</span> <span class="pl-c1">in</span> <span class="pl-s1">row</span>[<span class="pl-s">'x'</span>] <span class="pl-k">if</span> <span class="pl-s1">el</span> <span class="pl-c1">in</span> <span class="pl-s1">row</span>[<span class="pl-s">'y'</span>]], <span class="pl-s1">axis</span><span class="pl-c1">=</span><span class="pl-c1">1</span>)</pre></div>
<h4 dir="auto">Problem description</h4>
<p dir="auto">When a <code class="notranslate">DataFrame</code> has a <code class="notranslate">MultiIndex</code>, and the function passed to <code class="notranslate">DataFrame.apply</code> returns all <code class="notranslate">list</code>s, weird stuff happens and an unintelligible error occurs.</p>
<p dir="auto">What ends up happening is that the result somehow gets coerced to a list of arrays (not sure where or why the list-&gt;array conversion happens), and then submitted to <code class="notranslate">DataFrame.__init__</code>, which tries to massage it that into a DataFrame, and fails.</p>
<p dir="auto">Resulting error: <code class="notranslate">ValueError: Empty data passed with indices specified.</code>, emitted from deep within the bowels of <code class="notranslate">pandas/core/internals.py</code>, specifically  <code class="notranslate">create_block_manager_from_arrays</code>.</p>
<p dir="auto">This happens regardless of what the <code class="notranslate">reduce=</code> argument is set to.</p>
<h4 dir="auto">Expected Output</h4>
<p dir="auto">Don't try to manipulate the output. Return a <code class="notranslate">Series</code> of <code class="notranslate">list</code>s.</p>
<p dir="auto">In the example above, that'd be:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="pd.Series([[], ['q']], index=df.index)"><pre class="notranslate"><span class="pl-s1">pd</span>.<span class="pl-v">Series</span>([[], [<span class="pl-s">'q'</span>]], <span class="pl-s1">index</span><span class="pl-c1">=</span><span class="pl-s1">df</span>.<span class="pl-s1">index</span>)</pre></div>
<h4 dir="auto">Output of <code class="notranslate">pd.show_versions()</code></h4>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="INSTALLED VERSIONS
------------------
commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-514.26.2.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.1
pytest: None
pip: 9.0.1
setuptools: 36.6.0
Cython: None
numpy: 1.12.1
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.9999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None"><pre class="notranslate"><code class="notranslate">INSTALLED VERSIONS
------------------
commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Linux
OS-release: 3.10.0-514.26.2.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.1
pytest: None
pip: 9.0.1
setuptools: 36.6.0
Cython: None
numpy: 1.12.1
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 2.0.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.9999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</code></pre></div>
<h4 dir="auto">Code Sample, a copy-pastable example if possible</h4>
<p dir="auto">import pandas as pd</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="#Create sample dataframe
test = pd.DataFrame({&quot;a&quot;:[&quot;12&quot;, float(&quot;nan&quot;), &quot;3&quot;, &quot;meow&quot; ]})

# Select rows that are null or numeric:
print len(test[ test[&quot;a&quot;].isnull() | test[&quot;a&quot;].str.isnumeric()])

# Select rows that are numeric or null
print len(test[ test[&quot;a&quot;].str.isnumeric() | test[&quot;a&quot;].isnull() ])

# These two should return the same result, right? Unfortunately they don't"><pre class="notranslate"><code class="notranslate">#Create sample dataframe
test = pd.DataFrame({"a":["12", float("nan"), "3", "meow" ]})

# Select rows that are null or numeric:
print len(test[ test["a"].isnull() | test["a"].str.isnumeric()])

# Select rows that are numeric or null
print len(test[ test["a"].str.isnumeric() | test["a"].isnull() ])

# These two should return the same result, right? Unfortunately they don't
</code></pre></div>
<p dir="auto">Notice that <code class="notranslate">Series.str.isnumeric</code> throws an exception on null entries. However, I believe that this may introduce bugs silently</p>
<h4 dir="auto">Expected Output</h4>
<p dir="auto">Expected output is:<br>
2<br>
2</p>
<p dir="auto">Instead, I get:<br>
3<br>
2</p>
<h4 dir="auto">output of <code class="notranslate">pd.show_versions()</code></h4>
<h2 dir="auto">INSTALLED VERSIONS</h2>
<p dir="auto">commit: None<br>
python: 2.7.12.final.0<br>
python-bits: 64<br>
OS: Darwin<br>
OS-release: 15.6.0<br>
machine: x86_64<br>
processor: i386<br>
byteorder: little<br>
LC_ALL: None<br>
LANG: None</p>
<p dir="auto">pandas: 0.18.1</p>