py.test to test Cython C API modules
Question: I'm trying to set up unit tests for a Cython module to test some functions
that do not have python interface. The first idea was to check if `.pyx` files
could directly be used by `py.test`'s test runner, but apparently it only
scans for `.py` files.

Second idea was to write the `test_*` methods in one Cython module, which
could then be imported into a plain `.py` file. Let say we have a `foo.pyx`
module with the contents we want to test:

    
    
    cdef int is_true():
        return False
    

then a `test_foo.pyx` module that uses the C API to test the `foo` module:

    
    
    cimport foo
    
    def test_foo():
        assert foo.is_true()
    

and then import these in a plain `cython_test.py` module that would just
contain this line:

    
    
    from foo_test import *
    

The `py.test` test runner does find `test_foo` this way, but then reports:

    
    
    /usr/lib/python2.7/inspect.py:752: in getargs
        raise TypeError('{!r} is not a code object'.format(co))
    E   TypeError: <built-in function test_foo> is not a code object
    

Is there any better way to test Cython C-API code using `py.test`?


Answer: So, in the end, I managed to get `py.test` to run tests directly from a
Cython-compiled `.pyx` files. However, the approach is a terrible hack devised
to make use of `py.test` Python test runner as much as possible. It might stop
working with any `py.test` version different from the one I prepared the hack
to work with (which is 2.7.2).

First thing was to defeat `py.test`'s focus on `.py` files. Initially,
`py.test` refused to import anything that didn't have a file with `.py`
extension. Additional problem was that `py.test` verifies whether the module's
`__FILE__` matches the location of the `.py` file. Cython's `__FILE__`
generally does not contain the name of the source file though. I had to
override this check. I don't know if this override breaks anything—all I can
say is that the tests seem to run well, but if you're worried, please consult
your local `py.test` developer. This part was implemented as a local
`conftest.py` file.

    
    
    import _pytest
    import importlib
    
    
    class Module(_pytest.python.Module):
        # Source: http://stackoverflow.com/questions/32250450/
        def _importtestmodule(self):
            # Copy-paste from py.test, edited to avoid throwing ImportMismatchError.
            # Defensive programming in py.test tries to ensure the module's __file__
            # matches the location of the source code. Cython's __file__ is
            # different.
            # https://github.com/pytest-dev/pytest/blob/2.7.2/_pytest/python.py#L485
            path = self.fspath
            pypkgpath = path.pypkgpath()
            modname = '.'.join(
                [pypkgpath.basename] +
                path.new(ext='').relto(pypkgpath).split(path.sep))
            mod = importlib.import_module(modname)
            self.config.pluginmanager.consider_module(mod)
            return mod
    
        def collect(self):
            # Defeat defensive programming.
            # https://github.com/pytest-dev/pytest/blob/2.7.2/_pytest/python.py#L286
            assert self.name.endswith('.pyx')
            self.name = self.name[:-1]
            return super(Module, self).collect()
    
    
    def pytest_collect_file(parent, path):
        # py.test by default limits all test discovery to .py files.
        # I should probably have introduced a new setting for .pyx paths to match,
        # for simplicity I am hard-coding a single path.
        if path.fnmatch('*_test.pyx'):
            return Module(path, parent)
    

Second major problem is that `py.test` uses Python's `inspect` module to check
names of function arguments of unit tests. Remember that `py.test` does that
to inject fixtures, which is a pretty nifty feature, worth preserving.
`inspect` does not work with Cython, and in general there seems to be no easy
way to make original `inspect` to work with Cython. Nor there is any other
good way to inspect Cython function's list of arguments. For now I decided to
make a small workaround where I'm wrapping all test functions in a pure Python
function with desired signature.

In addition to that, it seems that Cython automatically puts a `__test__`
attribute to each `.pyx` module. The way Cython does that interferes with
`py.test`, and needed to be fixed. As far as I know, `__test__` is an internal
detail of Cython not exposed anywhere, so it should not matter that we're
overwriting it. In my case, I put the following function into a `.pxi` file
for inclusion in any `*_test.pyx` file:

    
    
    from functools import wraps
    
    
    # For https://github.com/pytest-dev/pytest/blob/2.7.2/_pytest/python.py#L340
    # Apparently Cython injects its own __test__ attribute that's {} by default.
    # bool({}) == False, and py.test thinks the developer doesn't want to run
    # tests from this module.
    __test__ = True
    
    
    def cython_test(signature=""):
        ''' Wrap a Cython test function in a pure Python call, so that py.test
        can inspect its argument list and run the test properly.
    
        Source: http://stackoverflow.com/questions/32250450/'''
        if isinstance(signature, basestring):
            code = "lambda {signature}: func({signature})".format(
                signature=signature)
    
            def decorator(func):
                return wraps(func)(eval(code, {'func': func}, {}))
    
            return decorator
    
        # case when cython_test was used as a decorator directly, getting
        # a function passed as `signature`
        return cython_test()(signature)
    

After that, I could implement tests like:

    
    
    include "cython_test_helpers.pxi"
    from pytest import fixture
    
    cdef returns_true():
        return False
    
    @cython_test
    def test_returns_true():
        assert returns_true() == True
    
    @fixture
    def fixture_of_true():
        return True
    
    @cython_test('fixture_of_true')
    def test_fixture(fixture_of_true):
        return fixture_of_true == True
    

If you decide to use the hack described above, please remember to leave
yourself a comment with a link to this answer—I'll try to keep it updated in
case better solutions are available.

