<p dir="auto">When using <code class="notranslate">np.frombuffer</code> or <code class="notranslate">np.ndarray(..., buffer=...)</code>, the object supporting the buffer interface is passed through unchanged. However, this makes it tricky for the base object to know if it's still being in use. <code class="notranslate">mmap.mmap</code> keeps a count of <code class="notranslate">exports</code>, and will refuse to close if the internal buffer is in use somewhere. Since <code class="notranslate">numpy</code> doesn't store this as a <code class="notranslate">memoryview</code>, the export count isn't incremented and can result in a segfault:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [1]: import mmap, numpy as np

In [2]: buffer = mmap.mmap(-1, 100)

In [3]: x = np.frombuffer(buffer, dtype='i1')

In [4]: x[:5]
Out[4]: array([0, 0, 0, 0, 0], dtype=int8)

In [5]: buffer.close()  # close the buffer

In [6]: x[:5]
Out[6]: Segmentation fault: 11"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">1</span>]: <span class="pl-s1">import</span> <span class="pl-s1">mmap</span>, <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>

<span class="pl-v">In</span> [<span class="pl-c1">2</span>]: <span class="pl-s1">buffer</span> <span class="pl-c1">=</span> <span class="pl-s1">mmap</span>.<span class="pl-en">mmap</span>(<span class="pl-c1">-</span><span class="pl-c1">1</span>, <span class="pl-c1">100</span>)

<span class="pl-v">In</span> [<span class="pl-c1">3</span>]: <span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">frombuffer</span>(<span class="pl-s1">buffer</span>, <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s">'i1'</span>)

<span class="pl-v">In</span> [<span class="pl-c1">4</span>]: <span class="pl-s1">x</span>[:<span class="pl-c1">5</span>]
<span class="pl-v">Out</span>[<span class="pl-c1">4</span>]: <span class="pl-en">array</span>([<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>], <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">int8</span>)

<span class="pl-v">In</span> [<span class="pl-c1">5</span>]: <span class="pl-s1">buffer</span>.<span class="pl-en">close</span>()  <span class="pl-c"># close the buffer</span>

<span class="pl-v">In</span> [<span class="pl-c1">6</span>]: <span class="pl-s1">x</span>[:<span class="pl-c1">5</span>]
<span class="pl-v">Out</span>[<span class="pl-c1">6</span>]: <span class="pl-v">Segmentation</span> <span class="pl-s1">fault</span>: <span class="pl-c1">11</span></pre></div>
<p dir="auto">If stored as a <code class="notranslate">memoryview</code>:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [1]: import mmap, numpy as np

In [2]: buffer = mmap.mmap(-1, 100)

In [3]: x = np.frombuffer(memoryview(buffer), dtype='i1')

In [4]: x[:5]
Out[4]: array([0, 0, 0, 0, 0], dtype=int8)

In [5]: x.base
Out[5]: &lt;memory at 0x105139d08&gt;

In [6]: buffer.close()
---------------------------------------------------------------------------
BufferError                               Traceback (most recent call last)
&lt;ipython-input-6-9666a5d75f9e&gt; in &lt;module&gt;()
----&gt; 1 buffer.close()

BufferError: cannot close exported pointers exist"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">1</span>]: <span class="pl-s1">import</span> <span class="pl-s1">mmap</span>, <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>

<span class="pl-v">In</span> [<span class="pl-c1">2</span>]: <span class="pl-s1">buffer</span> <span class="pl-c1">=</span> <span class="pl-s1">mmap</span>.<span class="pl-en">mmap</span>(<span class="pl-c1">-</span><span class="pl-c1">1</span>, <span class="pl-c1">100</span>)

<span class="pl-v">In</span> [<span class="pl-c1">3</span>]: <span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">frombuffer</span>(<span class="pl-en">memoryview</span>(<span class="pl-s1">buffer</span>), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s">'i1'</span>)

<span class="pl-v">In</span> [<span class="pl-c1">4</span>]: <span class="pl-s1">x</span>[:<span class="pl-c1">5</span>]
<span class="pl-v">Out</span>[<span class="pl-c1">4</span>]: <span class="pl-en">array</span>([<span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>, <span class="pl-c1">0</span>], <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">int8</span>)

<span class="pl-v">In</span> [<span class="pl-c1">5</span>]: <span class="pl-s1">x</span>.<span class="pl-s1">base</span>
<span class="pl-v">Out</span>[<span class="pl-c1">5</span>]: <span class="pl-c1">&lt;</span><span class="pl-s1">memory</span> <span class="pl-s1">at</span> <span class="pl-c1">0x105139d08</span><span class="pl-c1">&gt;</span>

<span class="pl-v">In</span> [<span class="pl-c1">6</span>]: <span class="pl-s1">buffer</span>.<span class="pl-en">close</span>()
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span>
<span class="pl-v">BufferError</span>                               <span class="pl-v">Traceback</span> (<span class="pl-s1">most</span> <span class="pl-s1">recent</span> <span class="pl-s1">call</span> <span class="pl-s1">last</span>)
<span class="pl-c1">&lt;</span><span class="pl-s1">ipython</span><span class="pl-c1">-</span><span class="pl-s1">input</span><span class="pl-c1">-</span><span class="pl-c1">6</span><span class="pl-c1">-</span><span class="pl-c1">9666</span><span class="pl-s1">a5d75f9e</span><span class="pl-c1">&gt;</span> <span class="pl-s1">in</span> <span class="pl-c1">&lt;</span><span class="pl-s1">module</span><span class="pl-c1">&gt;</span>()
<span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">-</span><span class="pl-c1">&gt;</span> <span class="pl-c1">1</span> <span class="pl-s1">buffer</span>.<span class="pl-en">close</span>()

<span class="pl-v">BufferError</span>: <span class="pl-s1">cannot</span> <span class="pl-s1">close</span> <span class="pl-s1">exported</span> <span class="pl-s1">pointers</span> <span class="pl-s1">exist</span></pre></div>
<p dir="auto">This affects <code class="notranslate">np.memmap</code> as well, but since <code class="notranslate">np.memmap</code> manages its own <code class="notranslate">mmap</code> object, closing that could be considered bad behavior.</p>
<p dir="auto">This crashes in python2 and python3 with a segfault:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import mmap

# create test file
a = arange(10., dtype='f4')
a.tofile('test')

# opens it as a memmap
with open('test', 'r+b') as f:
      mm = mmap.mmap(f.fileno(),0)

#view the mmap, then close it
x = np.frombuffer(mm, dtype='f4')
mm.close()
print(x)  # segfault"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">mmap</span>

<span class="pl-c"># create test file</span>
<span class="pl-s1">a</span> <span class="pl-c1">=</span> <span class="pl-en">arange</span>(<span class="pl-c1">10.</span>, <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s">'f4'</span>)
<span class="pl-s1">a</span>.<span class="pl-en">tofile</span>(<span class="pl-s">'test'</span>)

<span class="pl-c"># opens it as a memmap</span>
<span class="pl-k">with</span> <span class="pl-en">open</span>(<span class="pl-s">'test'</span>, <span class="pl-s">'r+b'</span>) <span class="pl-k">as</span> <span class="pl-s1">f</span>:
      <span class="pl-s1">mm</span> <span class="pl-c1">=</span> <span class="pl-s1">mmap</span>.<span class="pl-en">mmap</span>(<span class="pl-s1">f</span>.<span class="pl-en">fileno</span>(),<span class="pl-c1">0</span>)

<span class="pl-c">#view the mmap, then close it</span>
<span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">np</span>.<span class="pl-en">frombuffer</span>(<span class="pl-s1">mm</span>, <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s">'f4'</span>)
<span class="pl-s1">mm</span>.<span class="pl-en">close</span>()
<span class="pl-en">print</span>(<span class="pl-s1">x</span>)  <span class="pl-c"># segfault</span></pre></div>
<p dir="auto">In <code class="notranslate">frombuffer</code> mumpy uses the PEP3118 interface to the mmap to get a pointer to the data, so it seems we are using the interface incorrectly somewhere.</p>
<p dir="auto">It seems to me that the "correct" behavior may be for it to be impossible to close the memmap while pointers to it exist; this is the behavior for <code class="notranslate">memoryview</code>s of mmaps. That is, the line <code class="notranslate">mm.close()</code> shoud raise an error <code class="notranslate">BufferError: cannot close exported pointers</code>, as in currently happens in python3:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; import mmap
&gt;&gt;&gt; with open('test', 'r+b') as f:
&gt;&gt;&gt;     mm = mmap.mmap(f.fileno(),0)
&gt;&gt;&gt; mv = memoryview(mm)
&gt;&gt;&gt; mm.close()
BufferError: cannot close exported pointers exist"><pre class="notranslate"><span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">import</span> <span class="pl-s1">mmap</span>
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">with</span> <span class="pl-en">open</span>(<span class="pl-s">'test'</span>, <span class="pl-s">'r+b'</span>) <span class="pl-k">as</span> <span class="pl-s1">f</span>:
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span>     <span class="pl-s1">mm</span> <span class="pl-c1">=</span> <span class="pl-s1">mmap</span>.<span class="pl-en">mmap</span>(<span class="pl-s1">f</span>.<span class="pl-en">fileno</span>(),<span class="pl-c1">0</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">mv</span> <span class="pl-c1">=</span> <span class="pl-en">memoryview</span>(<span class="pl-s1">mm</span>)
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">mm</span>.<span class="pl-en">close</span>()
<span class="pl-v">BufferError</span>: <span class="pl-s1">cannot</span> <span class="pl-s1">close</span> <span class="pl-s1">exported</span> <span class="pl-s1">pointers</span> <span class="pl-s1">exist</span></pre></div>
<p dir="auto">however note also that in python2 we cannot create the memoryview (<code class="notranslate">mv</code>) because of an error <code class="notranslate">TypeError: cannot make memory view because object does not have the buffer interface</code>.</p>
<p dir="auto">Note how <a href="https://github.com/python/cpython/blob/b879fe82e7e5c3f7673c9a7fa4aad42bd05445d8/Modules/mmapmodule.c#L727">mmap_buffer_getbuf (link)</a> in the mmap module takes care to keep track of any buffer access to the mmap (the <code class="notranslate">exports</code> counter). Evidently we are somehow bypassing that in numpy.</p>