Python ctype - How to pass data between C functions
Question: I have a self-made C library that I want to access using python. The problem
is that the code consists essentially of two parts, an initialization to read
in data from a number of files and a few calculations that need to be done
only once. The other part is called in a loop and uses the data generated
before repeatedly. To this function I want to pass parameters from python.

My idea was to write two C wrapper functions, "init" and "loop" - "init" reads
the data and returns a void pointer to a structure that "loop" can use
together with additional parameters that I can pass on from python. Something
like

    
    
    void *init() {
      struct *mystruct ret = (mystruct *)malloc(sizeof(mystruct));
      /* Fill ret with data */
      return ret;
    }
    
    float loop(void *data, float par1, float par2) {
      /* do stuff with data, par1, par2, return result */
    }
    

I tried calling "init" from python as a c_void_p, but since "loop" changes
some of the contents of "data" and ctypes' void pointers are immutable, this
did not work.

Other solutions to similar problems I saw seem to require knowledge of how
much memory "init" would use, and I do not know that.

Is there a way to pass data from one C function to another through python
without telling python exactly what or how much it is? Or is there another way
to solve my problem?

* * *

I tried (and failed) to write a minimum crashing example, and after some
debugging it turned out there was a bug in my C code. Thanks to everyone who
replied! Hoping that this might help other people, here is a sort-of-minimal
working version (still without separate 'free' - sorry):

pybug.c:

    
    
    #include <stdio.h>
    #include <stdlib.h>
    
    typedef struct inner_struct_s {
      int length;
      float *array;
    } inner_struct_t;
    
    typedef struct mystruct_S {
      int id;
      float start;
      float end;
      inner_struct_t *inner;
    } mystruct_t;
    
    void init(void **data) {
      int i;
      mystruct_t *mystruct = (mystruct_t *)malloc(sizeof(mystruct_t));
      inner_struct_t *inner = (inner_struct_t *)malloc(sizeof(inner_struct_t));
      inner->length = 10;
      inner->array = calloc(inner->length, sizeof(float));
      for (i=0; i<inner->length; i++)
        inner->array[i] = 2*i;
      mystruct->id = 0;
      mystruct->start = 0;
      mystruct->end = inner->length;
      mystruct->inner = inner;
      *data = mystruct;
    }
    
    float loop(void *data, float par1, float par2, int newsize) {
      mystruct_t *str = data;
      inner_struct_t *inner = str->inner;
      int i;
      inner->length = newsize;
      inner->array  = realloc(inner->array, newsize * sizeof(float));
      for (i=0; i<inner->length; i++)
        inner->array[i] = par1 + i * par2;
      return inner->array[inner->length-1];
    }
    

compile as

    
    
    cc -c -fPIC pybug.c
    cc -shared -o libbug.so pybug.o
    

Run in python:

    
    
    from ctypes import *
    sl = CDLL('libbug.so')
    # What arguments do functions take / return?
    sl.init.argtype = c_void_p
    sl.loop.restype = c_float
    sl.loop.argtypes = [c_void_p, c_float, c_float, c_int]
    
    # Init takes a pointer to a pointer
    px = c_void_p()
    sl.init(byref(px))
    
    # Call the loop a couple of times
    for i in range(10):
        print sl.loop(px, i, 5, 10*i+5)
    


Answer: You should have a corresponding function to `free` the data buffer when the
caller is done. Otherwise I don't see the issue. Just pass the pointer to
`loop` that you get from `init`.

    
    
    init.restype = c_void_p
    loop.argtypes = [c_void_p, c_float, c_float]
    loop.restype = c_float
    

I'm not sure what you mean by "ctypes' void pointers are immutable", unless
you're talking about `c_char_p` and `c_wchar_p`. The issue there is if you
pass a Python string as an argument it uses Python's private pointer to the
string buffer. If a function can change the string, you should first copy it
to a `c_char` or `c_wchar` array.

Here's a simple example showing the problem of passing a Python string (2.x
byte string) as an argument to a function that modifies it. In this case it
changes index 0 to '\x00':

    
    
    >>> import os
    >>> from ctypes import *
    >>> open('tmp.c', 'w').write("void f(char *s) {s[0] = 0;}")
    >>> os.system('gcc -shared -fPIC -o tmp.so tmp.c')
    0
    >>> tmp = CDLL('./tmp.so')
    >>> tmp.f.argtypes = [c_void_p]
    >>> tmp.f.restype = None
    >>> tmp.f('a')
    >>> 'a'
    '\x00'
    >>> s = 'abc' 
    >>> tmp.f(s)
    >>> s
    '\x00bc'
    

This is specific to passing Python strings as arguments. It isn't a problem to
pass pointers to data structures that are intended to be mutable, either
ctypes data objects such as a `Structure`, or pointers returned by libraries.

