os.walk listing directory I use this script to get a list of all my photos in a folder and its subfolders. However, the program seems to be really slow.

There are 50,000 .jpg pictures in this folder and subfolders. I can reduce the for loops but the program still runs more or less at the same speed.

I am open to using lambda as well but would like to use Python's basic functions at the best possible speed. Can anyone suggest an improvement?

import os
from fnmatch import fnmatch
import sys

root = "C:\\Users\\Agartha\\Desktop\\photos"
pattern = "*.jpg"

with open("./files\\list.txt", "w") as a:
    for path, subdirs, files in os.walk(root):
       for filename in files:
            if fnmatch(filename,pattern):
                a.write(str(os.path.join(filename)) + '\n')
Here you do 4 things at the same time. Your are 


Walking over all the files of the directory
Comparing the filename to a pattern
Formatting the filename
Appending the filename to a file


To find out what is the slower step, you should decouple this

def find_files(root):
    for path, subdirs, files in os.walk(root):
        for filename in files:
            yield filename

def filter_filename(files, pattern):
    for filename in files:
        if fnmatch(filename,pattern):
            yield filename

def format_filenames(files, root):
    for filename in files:
        yield str(os.path.join(filename)) + '\n'

def writelines(out_file, files):
    for filename in files:
        out_file.write(filename)
    # or:
    # out_file.writelines(files)

def main(root, pattern, filename_out):
    files = find_files(root)
    files_filtered = filter_filename(files, pattern)
    files_formatted = format_filenames(files, root)
    with open(filename_out, 'w') as out_file:
        writelines(out_file, files_formatted)


Then Afterwards, there are 4 things you can time individually to find out the largest culprit in time consumption and work on that one:


files = list(find_files(root))
files_filtered = list(filter_filename(files, pattern))
files_formatted = list(format_filenames(files, root))
with open(filename_out, 'w') as out_file: writelines(out_file, files_formatted)