Test with imperative xfail in py.test always reports as xfail even if the passes
Question: I always thought that imperative and declarative usage of xfail/skip in
py.test should work in the same way. In the meantime I've noticed that if I
write a test that contains an imperative skip the result of the test will
always be "xfail" even it the test passes.

Here's some code:

    
    
    import pytest
    
    def test_should_fail():
        pytest.xfail("reason")
    
    @pytest.mark.xfail(reason="reason")
    def test_should_fail_2():
        assert 1
    

Running these tests will always result in:

    
    
    ============================= test session starts ==============================
    platform win32 -- Python 2.7.3 -- pytest-2.3.5 -- C:\Python27\python.exe
    collecting ... collected 2 items
    
    test_xfail.py:3: test_should_fail xfail
    test_xfail.py:6: test_should_fail_2 XPASS
    
    ===================== 1 xfailed, 1 xpassed in 0.02 seconds =====================
    

If I understand correctly what is written in the [user
manual](http://pytest.org/latest/skipping.html), both test should be
"XPASS'ed".

Is this a bug in py.test or am I getting something wrong?


Answer: When using the `pytest.xfail()` helper function you are effectively raising an
exception in the test function. Only when you are using the marker it is
possible for py.test to execute the test fully and give you an XPASS.

