ctypes: Correctly sublcass c_void_p for passing and returning custom data types, by example
Question: I am working with `ctypes` and cannot seem to figure out how to work with
custom data types. The hope is to have a Python interface to the public
methods of a C++ `cell` class and a C++ `cellComplex` class.

My current problem is working with the C function called
`get_elementsAtSpecifiedDim()`, defined below under `extern "C" {...`. This
function has been written to return a `void *` which is _really_ a
`std:vector< cell<double>* >`, a vector of pointers to C++ `cell` objects.

My Client code below shows an example of a call to this function (via the
Python `CellComplex` method `elemen()`) and this appears to me to be working
fine. You will notice that my Python implementation of `elemen()` declares the
return type `lib.elemen.restype` to be an array of (Python) `Cell` objects.
That's what I get, `>>>print cmplx.elemen()` yields

`[<cellComplex_python.Cell_Array_8 object at 0x10d5f59e0>,
<cellComplex_python.Cell_Array_12 object at 0x10d5f5830>,
<cellComplex_python.Cell_Array_6 object at 0x10d5f58c0>,
<cellComplex_python.Cell_Array_1 object at 0x10d5f5950>]`

**But here's the problem:**

Now I want to call my C functions on one of the `Cell` objects in an array in
this list. For example, `cmplx.elemen()[0][0]` is a `<Cell object at
0x10d5f5b00>`, so in my mind I should be able to do this:

`cmplx.elemen()[0][0].dim()`, **but this segfaults**.

My suspicion is that I am not creating the custom Python classes `Cell` and
`CellComplex` correctly. In particular, in Python class `Cell`, method
`dim(self):` I have the line `lib.dim.argtypes = [Cell]`, which must be
absolutely bogus. As well, I have this silly Python class `c_cellComplex`
which does nothing except allow me to indicate to myself what a particular
`ctypes.c_void_p` is supposed to point to. In fact, I claim that my
definitions of these Python classes are entirely bogus and I am being tricked
into thinking I am on the right track by the miracle that this runs at all (up
until I try to call a `Cell` method on a supposed `Cell` instance...

Client code:

    
    
    p = [[0,1],[0,1]] 
    cmplx = cellComplex(p)
    e = cmplx.elemen()
    
    e[0][0].dim() # segfault
    

**begin EDIT** [eryksun's answer
below](http://stackoverflow.com/a/20818377/390433) provides an example of how
to subclass c_void_p, and addresses a few other conceptual issues - start
there if you have the same questions I had.

The segfault issue comes from the fact that `get_elementsAtSpecifiedDim()`
defined in `extern C { ...` returns a memory address to a
`std::vector<cell<double>* >`, a datatype that cannot be parsed back in
Python. In this case I can just grab the pointers in the vector and return
them, like so:

    
    
    extern "C" {
    
      void * get_elementAtSpecifiedDimAndLoc(void *ptr, int dim, int nr) {
        cellComplex<double>* cmplx = static_cast<cellComplex<double>* >(ptr);
        cell<double>* c = cmplx->elemen()[dim][nr];
        return c;
      }
    }
    

and can be called like so:

    
    
    def elemen(self):    
         el = []
         for i in range(self.dim):
              size  = lib.size_elementsAtSpecifiedDim(self,i)               
              cells = []
              for j in range(size):
                   cells.append(lib.get_elementAtSpecifiedDimAndLoc(self,i,j))            
              el.append(cells)
         return el
    # put this somewhere
    lib.get_elementAtSpecifiedDimAndLoc.restype  = Cell 
    lib.get_elementAtSpecifiedDimAndLoc.argtypes = [CellComplex,c_int,c_int]
    

**The client code now works.**

**end EDIT**

Here is the magnificent folly:

    
    
    # cellComplex_python.py 
    lib = ctypes.cdll.LoadLibrary('./cellComplex_lib.so')
    
    class c_cellComplex(ctypes.c_void_p):
         pass
    
    class Cell(ctypes.c_void_p):
    
         def dim(self):
              lib.dim.restype  = ctypes.c_int
              lib.dim.argtypes = [Cell]
              self.dimension = lib.dim(self)
              return self.dimension
    
    class CellComplex(ctypes.c_void_p):
    
         def __init__(self,p):   
              self.dimension = len(p)
              lib.new_cellComplex.restype  = c_cellComplex
              lib.new_cellComplex.argtypes = [(ctypes.c_double*2)*self.dimension, 
                                               ctypes.c_size_t]
              e = [(ctypes.c_double*2)(p[i][0],p[i][1]) for i in range(self.dimension)]
              point = ((ctypes.c_double*2)*self.dimension)(*e)
              self.cmplx = lib.new_cellComplex(point,self.dimension)
    
    
         def elemen(self):
              lib.size_elementsAtSpecifiedDim.restype  = ctypes.c_int
              lib.size_elementsAtSpecifiedDim.argtypes = [c_cellComplex,
                                                          ctypes.c_int]
              lib.get_elementsAtSpecifiedDim.argtypes  = [c_cellComplex,ctypes.c_int]
    
              self.sizeAtDim = []
              self.elements  = []
              for i in range(self.dimension+1):
    
                   self.sizeAtDim.append(lib.size_elementsAtSpecifiedDim(self.cmplx,i))
                   lib.get_elementsAtSpecifiedDim.restype = Cell*self.sizeAtDim[i]
                   self.elements.append(lib.get_elementsAtSpecifiedDim(self.cmplx,i))
    
              return self.elements
    

`C` code:

    
    
    // cellComplex_extern.cpp
    #include<"cell.hpp">
    #include<"cellComplex.hpp"> 
    
    extern "C" {
    
      void * new_cellComplex(double p[][2], size_t dim) {
        std::vector< std::pair<double,double> > point;
        for (size_t i=0; i<dim; ++i) {
          point.push_back( std::make_pair(p[i][0],p[i][1]));
        }   
        cellComplex<double>* cmplx = new cellComplex<double>(point);
        return cmplx;
      }
    
      void * get_elementsAtSpecifiedDim(void *ptr, int dim) {
        cellComplex<double>* cmplx = static_cast<cellComplex<double>* >(ptr);
        std::vector<std::vector<cell<double>* > >* e = &cmplx->elemen();
        return &e[dim];
      }
    
      int size_elementsAtSpecifiedDim(void *ptr, int dim) {
        cellComplex<double>* cmplx = static_cast<cellComplex<double>* >(ptr);
        return cmplx->elemen()[dim].size();
      }  
    
      int dim(void *ptr) {
        cell<double>* ref = static_cast<cell<double>* >(ptr);
        return ref->dim();
      }
    
    }
    


Answer: Instead of subclassing `c_void_p`, you can define the class method
`from_param` and instance attribute `_as_parameter_`. You may not need either
option if you're just proxying a C++ object that's referenced in a private
attribute such as `_obj`. That said, a subclass of `c_void_p` can be used
directly with ctypes pointers, arrays, and structs, which may be convenient in
your overall design.

The following example may help:

    
    
    from ctypes import *
    
    __all__ = ['CellComplex']
    
    class Cell(c_void_p):
    
        def __new__(cls, *args, **kwds):
            raise TypeError("cannot create %r instances" % cls.__name__)
    
        @property
        def dimension(self):
            return lib.dim(self)
    
    class CellComplex(c_void_p):
    
        def __init__(self, p):
            pair = c_double * 2
            point = (pair * len(p))(*(pair(*q[:2]) for q in p))
            self.value = lib.new_cellComplex(point, len(p)).value
    
        @property
        def dimension(self):
            """Wrap a function that returns size_t."""
            return lib.????????(self)
    
        def get_elements(self):
            el = []
            for i in range(self.dimension):
                size = lib.size_elementsAtSpecifiedDim(self, i)
                cells = lib.get_elementsAtSpecifiedDim(self, i)
                el.append(cells[:size])
            return el
    

Function pointer definitions:

    
    
    lib = CDLL('./cellComplex_lib.so')
    
    lib.dim.restype = c_int
    lib.dim.argtypes = [Cell]
    
    lib.new_cellComplex.restype = CellComplex
    lib.new_cellComplex.argtypes = [POINTER(c_double * 2), c_size_t]
    
    lib.size_elementsAtSpecifiedDim.restype = c_int
    lib.size_elementsAtSpecifiedDim.argtypes = [CellComplex, c_int]
    
    lib.get_elementsAtSpecifiedDim.restype = POINTER(Cell)
    lib.get_elementsAtSpecifiedDim.argtypes = [CellComplex, c_int]
    

I separated out the function pointer definitions from the class method
definitions. There's no need to redefine a function pointer's `restype` and
`argtypes` every time a method is called. If the function returns a variably
sized array, you're better off setting it to a pointer type. You can slice the
result to a list or `cast` to an array type.

`CellComplex` is initialized by a sequence `p` of floating point pairs such as
`[[0.1, 0.2], [0.3, 0.4], [0.5, 0.6]]`.

I eliminated the `c_cellComplex` class. You can just set the instance's
`value`.

`lib.new_cellComplex` returns an instance of `CellComplex`, but ctypes
bypasses `__new__` and `__init__` when `CellComplex` is used as a `restype`,
so that's not an issue. It would be less twisted to instead override
`__new__`, but you'd still have to override `c_void_p.__init__`.

The `dimension` attribute needs to be a property that calls an exported
function, instead of relying on static data in the Python object.

