How do closures work in runpy?
Question: I get unexpected behaviour when I try to run methods defined in a file loaded
using the [runpy](http://docs.python.org/library/runpy.html) module. The
methods do not see any variables (including imported modules) defined outside
of that method. Here is how I am doing it:

    
    
    #test.py
    import runpy
    env = runpy.run_path('test', {'y':'world'})
    env['fn']()
    

~

    
    
    #test
    import re
    
    print(re.compile(r'^hello', re.IGNORECASE).sub('', "hello world"))
    x = "hello"
    print(x)
    print(y)
    
    def fn():
        try:
            print(re.compile(r'^hello', re.IGNORECASE).sub('', "hello world"))
        except:
            print("No re")
        try:
            print(x)
        except:
            print("No x")
        try:
            print(y)
        except:
            print("No y")
    

My expected output of test.py would be:

    
    
     world
    hello
    world
     world
    hello
    world
    

because fn would form a closure for re, x and y.

However, instead I get:

    
    
     world
    hello
    world
    No re
    None
    None
    

It looks like re isn't defined within fn even though it should be with normal
closure behaviour. x and y are even stranger because they appear to be defined
but set to None.

Why is this and how do closures work with runpy? How can I achieve normal
behaviour such that fn can 'see' outside variables?


Answer: OK, this is a curiosity of the way Python handles modules, which I know about
but don't fully understand. I've come across it while working on IPython,
where it's explained in [a
comment](https://github.com/ipython/ipython/blob/master/IPython/core/interactiveshell.py#L894).

When Python runs a module, it produces a module object, the attributes of
which are the global names in the module. When the module falls out of scope
and is being destroyed, these attributes are set to `None`. Code which was
defined in the function then sees these as the globals, as you found. You can
demonstrate this by adding `def g(): return globals()` to your file, then
calling `env["g"]()`.

I don't know if there's a way round this with `runpy`. IPython uses some
complicated code to reuse a module object for running other files, caching
copies of its `__dict__` to keep the references therein alive. Have a look at
the [`magic_run`
function](https://github.com/ipython/ipython/blob/master/IPython/core/magic.py#L1445)
if you're interested.

