Running doctests through iPython and pseudo-consoles
Question: I've got a fairly basic doctestable file:

    
    
    class Foo():
        """
        >>> 3+2
        5
        """
    
    if __name__ in ("__main__", "__console__"):
        import doctest
        doctest.testmod(verbose=True)
    

which works as expected when run directly through python.

However, in iPython, I get

    
    
    1 items had no tests:
        __main__
    0 tests in 1 items.
    0 passed and 0 failed.
    Test passed.
    

Since this is part of a Django project and will need access to all of the
appropriate variables and such that manage.py sets up, I can also run it
through a modified command, which uses code.InteractiveConsole, one result of
which is `__name__` gets set to '`__console__`'.

With the code above, I get the same result as with iPython. I tried changing
the last line to this:

    
    
     this = __import__(__name__)
     doctest.testmod(this, verbose=True)
    

and I get an ImportError on `__console__`, which makes sense, I guess. This
has no effect on either python or ipython.

So, I'd like to be able to run doctests successfully through all three of
these methods, especially the InteractiveConsole one, since I expect to be
needing Django pony magic fairly soon.

Just for clarification, this is what I'm expecting:

    
    
    Trying:
        3+2
    Expecting:
        5
    ok
    1 items had no tests:
        __main__
    1 items passed all tests:
       1 tests in __main__.Foo
    1 tests in 2 items.
    1 passed and 0 failed.
    Test passed.
    


Answer: The root problem is that `ipython` plays weird tricks with `__main__` (through
its own `FakeModule` module) so that, by the time `doctest` is introspecting
that "alleged module" through its `__dict__`, `Foo` is _NOT_ there -- so
doctest doesn't recurse into it.

Here's one solution:

    
    
    class Foo():
        """
        >>> 3+2
        5
        """
    
    if __name__ in ("__main__", "__console__"):
        import doctest, inspect, sys
        m = sys.modules['__main__']
        m.__test__ = dict((n,v) for (n,v) in globals().items()
                                if inspect.isclass(v))
        doctest.testmod(verbose=True)
    

This DOES produce, as requested:

    
    
    $ ipython dot.py 
    Trying:
        3+2
    Expecting:
        5
    ok
    1 items had no tests:
        __main__
    1 items passed all tests:
       1 tests in __main__.__test__.Foo
    1 tests in 2 items.
    1 passed and 0 failed.
    Test passed.
    Python 2.5.1 (r251:54863, Feb  6 2009, 19:02:12) 
      [[ snip snip ]]
    In [1]:
    

Just setting global `__test__` doesn't work, again because setting it as a
global of what you're thinking of as `__main__` does NOT actually place it in
the `__dict__` of the actual object that gets recovered by `m =
sys.modules['__main__']`, and the latter is exactly the expression `doctest`
is using internally (actually it uses `sys.modules.get`, but the extra
precaution is not necessary here since we do know that `__main__` exists in
`sys.modules`... it's just NOT the object you expect it to be!-).

Also, just setting `m.__test__ = globals()` directly does not work either, for
a different reason: `doctest` checks that the values in `__test__` are
strings, functions, classes, or modules, and without some selection you cannot
guarantee that `globals()` will satisfy that condition (in fact it won't).
Here I'm selecting just classes, if you also want functions or whatnot you can
use an `or` in the `if` clause in the genexp within the `dict` call.

I don't know exactly how you're running a Django shell that's able to execute
your script (as I believe `python manage.py shell` doesn't accept arguments,
you must be doing something else, and I can't guess exactly what!-), but a
similar approach should help (whether your Django shell is using ipython, the
default when available, or plain Python): appropriately setting `__test__` in
the object you obtain as `sys.modules['__main__']` (or `__console__`, if
that's what you're then passing on to doctest.testmod, I guess) should work,
as it mimics what doctest will then be doing internally to locate your test
strings.

And, to conclude, a philosophical reflection on design, architecture,
simplicity, transparency, and "black magic"...:

All of this effort is basically what's needed to defeat the "black magic" that
ipython (and maybe Django, though it may be simply delegating that part to
ipython) is doing on your behalf for your "convenience"... any time at which
two frameworks (or more;-) are independently doing each its own brand of black
magic, interoperability may suddenly require substantial effort and become
anything BUT convenient;-).

I'm not saying that the same convenience could have been provided (by any one
or more of ipython, django and/or doctests) _without_ black magic,
introspection, fake modules, and so on; the designers and maintainers of each
of those frameworks are superb engineers, and I expect they've done their
homework thoroughly, and are performing only the minimum amount of black magic
that's indispensable to deliver the amount of user convenience they decided
they needed. Nevertheless, even in such a situation, "black magic" suddenly
turns from a dream of convenience to a nightmare of debugging as soon as you
want to do something even marginally outside what the framework's author had
conceived.

OK, maybe in this case not quite a nightmare, but I do notice that this
question has been open a while and even with the lure of the bounty it didn't
get many answers yet -- though you now do have two answers to pick from, mine
using the `__test__` special feature of doctest, @codeape's using the peculiar
`__IP.magic_run` feature of ironpython. I prefer mine because it does not rely
on anything internal or undocumented -- `__test__` IS a documented feature of
doctest, while `__IP`, with those two looming leading underscores, scream
"deep internals, don't touch" to me;-)... if it breaks at the next point
release I wouldn't be at all surprised. Still, matter of taste -- that answer
may arguably be considered more "convenient".

But, this is exactly my point: convenience may come at an enormous price in
terms of giving up simplicity, transparency, and/or avoidance of
internal/undocumented/unstable features; so, as a lesson for all of us, the
least black magic &c we can get away with (even at the price of giving up an
epsilon of convenience here and there), the happier we'll all be in the long
run (and the happier we'll make other developers that need to leverage our
current efforts in the future).

