Is there an easy way to find which part of the code not closing the file
Question: I have a large program with large amount of code. And it s is opening the file
but not closing it.

### Question:

Is there an easy way to find out where this happening?

### More Details:

OS - Linux  
Python - 2.7

Why this is important? Imagine the situation:

    
    
    df -h 
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/sda5       157G   39G  110G  27% /
    

110 G Available. Let's create large file

    
    
    fallocate -l 10G large_file.csv
    

Now 100 G available

    
    
    df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/sda5       157G   49G  100G  34% /
    

lets'write the program that opens the file and run it:

    
    
    import time
    f = open('large_file.csv')
    try:
        while True:
            time.sleep(1)
    except:
        pass
    

While this is running, let's delete the file:

    
    
    rm large_file.csv
    

Checking space:

    
    
    df -h 
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/sda5       157G   49G  100G  34% /
    

You see it's still `100G` available.

So the question is how to easily find this type of issues in the large
program?


Answer: I remember reading somewhere that cpython is guaranteed to close all file
handles when garbage collecting, so unless you are segfaulting I guess your
python program is not the culprit (or there is a C module misbehaving and this
answer is worthless). MySQL is a known offender here (for deleting open file
handlers), so if there is a MySQL database involved I would bet on it.

That said, you can monkey-patch `__builtin__.open` as suggested by martineau
to raise an exception, catch the exception and use the inspect module to
search the traceback and check if the open call is inside a with statement or
try/finally block. The following example is very crude but I hope it can get
you started:

    
    
    #test.py
    import foo
    
    _old_open = open  # original function
    # monkey-patch
    def _new_open(*args, **kwargs):
        try:
            raise(Exception('dummy'))
        except Exception as e:
            import sys
            check_call(*sys.exc_info())
        return _old_open(*args, **kwargs)
    __builtins__.open = _new_open
    
    
    def check_call(e_type, e_value, tb):
        import inspect, sys
        # restore patch to avoid infinite recursion
        __builtins__.open = _old_open  
        try:
            stack = inspect.getouterframes(tb.tb_frame)
            frame_info = inspect.getframeinfo(stack[1][0])
            if frame_info.code_context[0].strip()\
                         .startswith('with '):
                return
            sys.stderr.write(
               "DEBUG: open call outside with block at "
               "{f.filename}, line {f.lineno}\n"
               .format(f=frame_info)
            )
        finally:
            __builtins__.open = _new_open
    
    
    if __name__ == '__main__':
        foo.baz('a.txt')
        foo.bar('a.txt')
    
    # foo.py
    def bar(fname):
        f = open(fname, 'w')
    
    def baz(fname):
        with open(fname, 'w') as f:
            f.write('dummy!')
    
    # result:
    # DEBUG: open call outside with block at 
    # /path/to/foo.py, line 13
    

