Python: Files stop being opened at a certain point
Question: I've written the following program in Python:

    
    
    import re
    import os
    import string
    
    folder = 'C:\Users\Jodie\Documents\Uni\Final Year Project\All Data'
    
    folderlisting = os.listdir(folder)
    for eachfolder in folderlisting:
        print eachfolder
        if os.path.isdir(folder + '\\' + eachfolder):
            filelisting = os.listdir('C:\Users\Jodie\Documents\Uni\Final Year Project\All Data\\' + eachfolder)
            print filelisting
    
            for eachfile in filelisting:
                if re.search('.genbank.txt$', eachfile):
                    genbankfile = open(eachfile, 'r')
                    print genbankfile
    
                if re.search('.alleles.txt$', eachfile):
                    allelesfile = open(eachfile, 'r')
                    print allelesfile
    

It looks through a lot of folders, and prints the following:

  1. The name of each folder, without the path
  2. A list of all files in each folder
  3. Two specific files in each folder (Any files containing ".genbank.txt" and ".alleles.txt").

The code works until it reaches a certain directory, and then fails with the
following error:

    
    
    Traceback (most recent call last):
      File "C:\Users\Jodie\Documents\Uni\Final Year Project\Altering Frequency Practice\Change_freq_data.py", line 16, in <module>
        genbankfile = open(eachfile, 'r')
    IOError: [Errno 2] No such file or directory: 'ABP1.genbank.txt'
    

The problem is:

  1. That file most definitely exists, since the program lists it before it tries to open the file.
  2. Even if I take that directory out of the original group of directories, the program throws up the same error for the next folder it iterates to. And the next, if that one is removed. And so on.

This makes me think that it's not the folder or any files in it, but some
limitation of Python? I have no idea. It has stumped me.

Any help would be appreciated.


Answer: You should use os.walk() <http://docs.python.org/library/os.html#os.walk>

Also, you need to read the contents of the file, you don't want to print the
file object. And you need to close the file when you're done or use a context
manager to close it for you.

would look something like:

    
    
    for root, dirs, files in os.walk(folder):
        for file_name in files:
            if re.search('.genbank.txt$', file_name) or \
                re.search('.alleles.txt$', file_name):
                with open(os.path.join(root, f), 'r') as f:
                    print f.read()
    

Keep in mind this is not 'exactly' what you're doing, this will walk the
entire tree, you may just want to walk a single level like you are already
doing.

