title: Cython : return a <object>PyObject*, will it leak?
body: <p>I am currently wrapping a library using Cython. For this purpose, I wanted to re-use one function of a pure-C binding.</p>
  
  <p>This is the basic setup:</p>
  
  <ul>
  <li>mylib.pxd</li>
  <li>old_lib.c</li>
  <li>old_lib.h</li>
  </ul>
  
  <p>In mylib.pxd I do :</p>
  
  <pre><code>cdef extern from old_lib.h:
      PyObject* get_pyobject()
  </code></pre>
  
  <p>And then pass <code>old_lib.c</code> as a source file in my extension:</p>
  
  <pre><code>setup(ext_modules=[Extension("mylib", sources=["mylib.pxd", "old_lib.c"])])
  </code></pre>
  
  <p>In mylib.pxd, I use <code>get_pyobject</code> that creates a new object that I want to return like so:</p>
  
  <pre><code>cdef PyObject* ptr
  ptr = get_pyobject()
  return &lt;object&gt;ptr
  </code></pre>
  
  <p>This gives me the desired behaviour, but I am afraid that this will leak the ptr reference.</p>
  
  <p>Will it ? I get confused because I found (old) references saying that you should manage PyObject* references by yourself and call Py_INCREF/DECREF accordingly but it seems that in the Cython <a href="https://github.com/cython/cython/wiki/FAQ#what-is-the-difference-between-pyobject-and-object" rel="nofollow">FAQ</a> they say :</p>
  
  <blockquote>
    <p>Note that the lifetime of the object is only bound to its owned references, not to any C pointers that happen to point to it.</p>
  </blockquote>
  
  <p>Does this mean that whenever the returned value is discarded, the <code>ptr</code> will be garbage collected?</p>
  
  <p>In the <code>old_lib.c</code> the flow goes like this:</p>
  
  <pre><code>PyObject* get_pyobject()
  {
       PyTypedObject* typeptr = PyObject_NEW_VAR(MyType, &amp;Type, size)
       fill_attribute(typeptr-&gt;attrib)
       return (PyObject*)typeptr
  }
  </code></pre>
  
  <p>Where <code>PyObject_NEW_VAR</code> is implemented in the python standard library (objimpl.h:196 in my version) using <code>PyObject_InitVar</code>. Thus, the returned reference is a <em>borrowed</em> reference, but as <code>PyObject_MALLOC</code> is used, I guess this is the only reference to this object. Relevant code:</p>
  
  <pre><code>#define PyObject_NEW_VAR(type, typeobj, n) \
  ( (type *) PyObject_InitVar( \
         (PyVarObject *) PyObject_MALLOC(_PyObject_VAR_SIZE((typeobj),(n)) ),\
         (typeobj), (n)) )
  </code></pre>
  
  <p>EDIT :
  I have checked, and when using the above code,<code>sys.getrefcount</code> returns 3. So as far as I understand, when I create the object, it comes with a refcount of 1. Then, when casting it to <code>object</code> its refcount is bumped to 2. It will thus never be garbage collected (unless there is a way to remove two refcounts to an object which has only one accessible pointer) and leak.
  If I insert a PY_DECREF, it still works and correctly returns 2. I also took time to rewrite that function directly in Cython, and it returns 2.</p>