segmentation fault - core dump in python C-extension
Question: I am writing a c-extension for python. As you can see below, the aim of the
code is to calculate the euclidean-dist of two vectors. the first param n is
the dimension of the vectors, the second , the third param is the two list of
float.

I call the function in python like this:

    
    
    import cutil
    cutil.c_euclidean_dist(2,[1.0,1,0],[0,0])
    

and it worked well, return the right result. but if i do it for more than 100
times(the dimension is 1*1000), it will cause segmentation fault - core dump:

    
    
    #!/usr/bin/env python
    #coding:utf-8
    import cutil
    import science
    import time
    a = []
    b = []
    d = 0.0 
    for x in range(2500):
        a.append([float(i+x) for i in range(1000)])
        b.append([float(i-x) for i in range(1000)])
    
    t1 = time.time()
    for x in range(500):
        d += cutil.c_euclidean_dist(1000,a[x],b[x])
    print time.time() - t1
    print d
    

the C code is here:

    
    
    #include <python2.7/Python.h>
    #include <math.h>
    
    static PyObject* cutil_euclidean_dist(PyObject* self, PyObject* args) {
        PyObject *seq_a, *seq_b;
        int n;
        float * array_a,* array_b;
        PyObject *item;
    
        PyArg_ParseTuple(args,"iOO", &n , &seq_a, &seq_b);
        if (!PySequence_Check(seq_a) || !PySequence_Check(seq_b)) {
           PyErr_SetString(PyExc_TypeError, "expected sequence");
           return NULL;
        }
    
        array_a =(float *)malloc(sizeof(float)*n);
        array_b =(float *)malloc(sizeof(float)*n);  
    
        if (NULL == array_a || NULL == array_b){
            PyErr_SetString(PyExc_TypeError, "malloc failed!");
            Py_DECREF(seq_a);
            Py_DECREF(seq_b);
            return NULL;
        }
    
        int i;
        for(i=0;i<n;i++){
            item = PySequence_GetItem(seq_a,i);
    
            if (!PyFloat_Check(item)) {
                free(array_a);  /* free up the memory before leaving */
                free(array_b);
                Py_DECREF(seq_a);
                Py_DECREF(seq_b);
                Py_DECREF(item);
                PyErr_SetString(PyExc_TypeError, "expected sequence of float");
                return NULL;
            }
            array_a[i] = PyFloat_AsDouble(item);
    
            Py_DECREF(item);
    
            item = PySequence_GetItem(seq_b,i);
            if(!PyFloat_Check(item)) {
                free(array_a);
                free(array_b);
                Py_DECREF(seq_a);
                Py_DECREF(seq_b);
                Py_DECREF(item);
                PyErr_SetString(PyExc_TypeError, "expected sequence of float"); 
                return NULL;
            }
            array_b[i] = PyFloat_AsDouble(item);
            Py_DECREF(item);
        }
    
        double sum = 0;
        for(i=0;i<n;i++){
            double delta = array_a[i] - array_b[i];
            sum += delta * delta;
        }
    
        free(array_a);
        free(array_b);
        Py_DECREF(seq_a);
        Py_DECREF(seq_b);
    
        return Py_BuildValue("d",sqrt(sum));
    }
    
    static PyMethodDef cutil_methods[] = {
        {"c_euclidean_dist",(PyCFunction)cutil_euclidean_dist,METH_VARARGS,NULL},
        {NULL,NULL,0,NULL}
    };
    
    PyMODINIT_FUNC initcutil(void) {
        Py_InitModule3("cutil", cutil_methods, "liurui's c extension for python");
    } 
    

the error msg:

    
    
    segmentation fault - core dump:
    

The c-extension is compiled to cutil.so, I do not know how to see the dump.
But i looked through my C code for many times,and can not find any problem..

May it be **a memory problem**?

It should be a very simple piece of C code, what's wrong with it? I need your
help~ thanks a lot !

here is the result of gdb /usr/bin/python2.7 ./core:

    
    
    root@ubuntu:/home/rrg/workspace/opencvTest/test# gdb /usr/bin/python2.7 ./core 
    GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
    Copyright (C) 2014 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "x86_64-linux-gnu".
    Type "show configuration" for configuration details.
    For bug reporting instructions, please see:
    <http://www.gnu.org/software/gdb/bugs/>.
    Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.
    For help, type "help".
    Type "apropos word" to search for commands related to "word"...
    Reading symbols from /usr/bin/python2.7...Reading symbols from /usr/lib/debug//usr/bin/python2.7...done.
    done.
    
    warning: core file may not match specified executable file.
    [New LWP 13787]
    [New LWP 13789]
    [New LWP 13790]
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
    Core was generated by `python py.py'.
    Program terminated with signal SIGSEGV, Segmentation fault.
    #0  0x00000000005398b3 in list_dealloc.16846 (op=0x7f688b2faa28) at ../Objects/listobject.c:309
    309 ../Objects/listobject.c: no such file or directory
    #0  0x00000000005398b3 in list_dealloc.16846 (op=0x7f688b2faa28) at ../Objects/listobject.c:309
    #1  0x00000000004fdb96 in insertdict_by_entry (value=<optimized out>, ep=0x1777fa8, hash=<optimized out>, key='b', mp=0x7f68a8dbb168) at ../Objects/dictobject.c:519
    #2  insertdict (value=<optimized out>, hash=<optimized out>, key='b', mp=0x7f68a8dbb168) at ../Objects/dictobject.c:556
    #3  dict_set_item_by_hash_or_entry (value=<optimized out>, ep=0x0, hash=<optimized out>, key='b', 
        op={'a': None, 'x': None, 'c': None, 'b': None, 'd': <float at remote 0x4480b30>, '__builtins__': <module at remote 0x7f68a8de6b08>, 'science': <module at remote 0x7f68a8ce4088>, '__package__': None, 'i': 999, 'cutil': <module at remote 0x7f68a8cdfbb0>, 'time': <module at remote 0x7f68a640ea28>, '__name__': '__main__', 't1': <float at remote 0xd012708>, '__doc__': None}) at ../Objects/dictobject.c:765
    #4  PyDict_SetItem (
        op=op@entry={'a': None, 'x': None, 'c': None, 'b': None, 'd': <float at remote 0x4480b30>, '__builtins__': <module at remote 0x7f68a8de6b08>, 'science': <module at remote 0x7f68a8ce4088>, '__package__': None, 'i': 999, 'cutil': <module at remote 0x7f68a8cdfbb0>, 'time': <module at remote 0x7f68a640ea28>, '__name__': '__main__', 't1': <float at remote 0xd012708>, '__doc__': None}, key=key@entry='b', 
        value=<optimized out>) at ../Objects/dictobject.c:818
    #5  0x000000000055a9e1 in _PyModule_Clear (m=<optimized out>) at ../Objects/moduleobject.c:139
    #6  0x00000000004f2ad4 in PyImport_Cleanup () at ../Python/import.c:473
    #7  0x000000000042fa89 in Py_Finalize () at ../Python/pythonrun.c:459
    #8  0x000000000046ac10 in Py_Main (argc=<optimized out>, argv=0x7fff3958d058) at ../Modules/main.c:665
    #9  0x00007f68a8665ec5 in __libc_start_main (main=0x46ac3f <main>, argc=2, argv=0x7fff3958d058, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff3958d048)
        at libc-start.c:287
    #10 0x000000000057497e in _start ()
    

Edit: I comment the last 2 sentences near the last return:

    
    
    Py_DECREF(seq_a);
    Py_DECREF(seq_b);  
    

and then it seems to work well. I feel very very strange... The purpose of the
two sentence is to free(or release) the two pyobject, why it works well
without the two sentences which I think are necessary?


Answer: > The c-extension is compiled to cutil.so, I do not know how to see the dump.

To solve this, I'm going to cite [GNU Radio's GDB/Python debugging mini-
tutorial](http://gnuradio.org/redmine/projects/gnuradio/wiki/TutorialsGDB):

> Luckily, there's a feature called core dumping that allows the state of your
> program to be stored in a file, allowing later analysis. Usually, that
> feature is disabled; you can enable it by:
>  
>  
>     ulimit -c unlimited
>  
>
> Note that this only works for processes spawned from the shell that you used
> ulimit in. What happens here is that the maximum size of a core dump is set
> to unlimited (the original value is 0 in most cases).
>
> Now, the core dump file lays in the current execution directory of the
> program that crashed. In our case, that's build/python/, but since all core
> dumps should have a name like core., we can use a little find magic:
>  
>  
>     marcus> find -type f -cmin 5 -name 'core.[0-9]*'
>  
>
> ./build/python/core.22608
>
> because that will find all _f_iles, changed/created within the last _5
> min_utes, having a name that matches.
>
> ## Using GDB with a core dump
>
> having found build/python/core.22608, we can now launch GDB:
>  
>  
>     gdb programname coredump
>  
>
> i.e.
>  
>  
>     gdb /usr/bin/python2 build/python/core.22608
>  
>
> A lot of information might scroll by.
>
> At the end, you're greeted by the GDB prompt:
>  
>  
>     (gdb)
>  
>
> ## Getting a backtrace
>
> Typically, you'd just get a backtrace (or shorter, bt). A backtrace is
> simply the hierarchy of functions that were called.
>  
>  
>      (gdb)bt
>  

[...] skipped,

> Frame #2 and following definitely look like they're part of the Python
> implementation -- that sounds bad, because GDB doesn't itself know how to
> debug python, but luckily, there's an extension to do that. So we can try to
> use py-bt:
>  
>  
>     (gdb) py-bt
>  
>
> If we get a undefined command error, we must stop here and make sure that
> the python development package is installed (python-devel on Redhatoids,
> python2.7-dev on Debianoids); for some systems, you should append the
> content of /usr/share/doc/{python-devel,python2.7-dev}/gdbinit[.gz] to your
> ~/.gdbinit, and re-start gdb.
>
> The output of py-bt now states clearly which python lines correspond to
> which stack frame (skipping those stack frames that are hidden to python,
> because they are in external libraries or python-implementation routines)
>
> ...

