title: Binding C array to Numpy array without copying
body: <p>I am writing a Python class that will wrap a C module containing a C struct. I am using the Cython language (a super-set language of Python and C). The C struct is malloc'd in the constructor and contains an array that I want to use in Python. The array will be represented in Python as a NumPy array but I don't want to copy the values to it. I want to link the NumPy array directly to the malloc'd memory. For this task I use the <a href="http://docs.scipy.org/doc/numpy/reference/c-api.array.html" rel="nofollow">NumPy Array API</a> and specifically this function:</p>
  
  <p><code>PyObject*</code><a href="http://docs.scipy.org/doc/numpy/reference/c-api.array.html#c.PyArray_SimpleNewFromData" rel="nofollow"><code>PyArray_SimpleNewFromData</code></a><code>(int nd, npy_intp* dims, int typenum, void* data)</code></p>
  
  <p>I managed to bind the NumPy array to the C struct's array using this code in Cython and it works well as long as the NumPy array and <code>MultimediaParams</code> object have the same lifetime:</p>
  
  <pre><code>cdef class MultimediaParams:
      def __init__(self, **kwargs):
          self._mm_np = &lt; mm_np *&gt; malloc(sizeof(mm_np))
          #some code...
  
      def as_ndarray(self): #TODO: what if self deallocated but numpy array still exists(segfault?)
          cdef numpy.npy_intp shape[1]
          cdef int arr_size = sizeof(self._mm_np[0].n2) / sizeof(self._mm_np[0].n2[0])
          shape[0] = &lt; numpy.npy_intp &gt; arr_size
          cdef numpy.ndarray ndarray
          ndarray = numpy.PyArray_SimpleNewFromData(1, shape, numpy.NPY_DOUBLE, self._mm_np[0].n2)
  
          return ndarray
  
      def __dealloc__(self):
          free(self._mm_np)
  </code></pre>
  
  <p>As you can see, the class has its <code>__dealloc__</code> method which will take care of the memory allocated in C and free it when there are no references to <code>MultimediaParams</code> instance.</p>
  
  <p>In this kind of binding NumPy is not owning the memory of the array.</p>
  
  <p><strong>The problem</strong>: when the <code>MultimediaParams</code> object is deallocated and the memory of the array is freed, the NumPy object is still pointing to memory that was just freed. This will cause a segfault when the NumPy object tries to access/modify the memory that was freed.</p>
  
  <p>How can I make sure the <code>MultimediaParams</code> object is not deallocated as long as there is a NumPy object using its memory?</p>
  
  <p>As I understand it, all I need to do is to make the NumPy object have a refference to a <code>MultimediaParams</code> instance from which it got the memory to point to.
  I tried to use <code>ndarray.base = &lt;PyObject*&gt;self</code> so NumPy will know its base object, this is supposed to add another reference to a <code>MultimediaParams</code> instance and will cause it not to be deallocated as long as the NumPy array is alive. This line causes my tests to fail because the contents of the NumPy array turn to garbage.</p>
  
  <p><strong>CLARIFICATION</strong>: <strong>The NumPy array does not take ownership of the C array memory and I don't want it to.</strong> I want <code>MultimediaParams</code> to be responsible for freeing the C struct (that contains the array data), but not to do it as long as the NumPy object is alive.</p>
  
  <p>Any suggestions?</p>