Converting malloc'ed buffers from C to Python without copy using Cython?
Question: In Cython, say I have a C function that returns a large buffer allocated with
malloc() and expected to be freed later with free().

Now I need to pass this buffer to Python as a (bytes) str object, which would
acquire ownership of it, and call free() later when the str object goes away.
Is this possible and how?


Answer: Have a look at <https://gist.github.com/1249305> for a implementation using
numpy.

If numpy is not an option then something like this could work, using a
memoryview:

    
    
    from libc.stdlib cimport free
    cimport fn # in here is the c function with signature: char * char_ret(int n);
    
    cdef class BuffWrap:
        cdef long siz
        cdef char * _buf
        cdef char[:] arr
        def __cinit__(self,  long siz):
            self.siz = siz
            self._buf = fn.char_ret(siz) # storing the pointer so it can be freed
            self.arr = <char[:siz]>self._buf
        def __dealloc__(self):
            free(self._buf)
            self.arr = None
        # here some extras:
        def __str__(self):
            if self.siz<11:
                return 'BuffWrap: ' + str([ii for ii in self.arr])
            else:
                return ('BuffWrap: ' + str([self.arr[ii] for ii in range(5)])[0:-1] + ' ... '
                        + str([self.arr[ii] for ii in range(self.siz-5, self.siz)])[1:])
        def __getitem__(self, ind): 
            """ As example of magic method.  Implement rest of to get slicing
            http://docs.cython.org/src/userguide/special_methods.html#sequences-and-mappings
            """
            return self.arr[ind]
    

Note the method `__dealloc__` which will free the memory held by the pointer
when all references to an instance of `BuffWrap` disappear and it gets garbage
collected. This automatic deallocation is a good reason to wrap the whole
thing in a class.

I couldn't figure out how one could use the returned pointer and use it for
the buffer of, say, a bytearray. If someone knows, I'd be interested to see.

