Python: My script will not allow me to create large files
Question: I'm making a small python script which will create random files in all shapes
and sizes but it will not let me create large files. I want to be able to
create files up to around 8GB in size, I know this would take a long amount of
time but I'm not concerned about that.

The problem is that Python 2.7 will not handle the large numbers I am throwing
at it in order to create the random text that will fill my files.

The aim of my code is to create files with random names and extentions, fill
the files with a random amount of junk text and save the files. It will keep
on repeating this until I close the command line window.

    
    
    import os
    import string
    import random
    
    
    ext = ['.zip', '.exe', '.txt', '.pdf', '.msi', '.rar', '.jpg', '.png', '.html', '.iso']
    
    min = raw_input("Enter a minimum file size eg: 112 (meaning 112 bytes): ")
    minInt = int(min)
    
    max = raw_input("Enter a maximum file size: ")
    maxInt = int(max)
    
    def name_generator(chars=string.ascii_letters + string.digits):
        return ''.join(random.choice(chars) for x in range(random.randint(1,10)))
    
    def text_generator(chars=string.printable + string.whitespace):
        return ''.join(random.choice(chars) for x in range(random.randint(minInt,maxInt)))
    
    def main():
        fileName = name_generator()
        extension = random.choice(ext)
        file = fileName + extension
    
        print 'Creating ==> ' + file
        fileHandle = open ( file, 'w' )
        fileHandle.write ( text_generator() )
        fileHandle.close()
        print file + ' ==> Was born!'
    
    while 1:
        main()
    

Any help will be much appreciated!


Answer: Make it lazy, as per the following:

    
    
    import string
    import random
    from itertools import islice
    
    chars = string.printable + string.whitespace
    # make infinite generator of random chars
    random_chars = iter(lambda: random.choice(chars), '')
    with open('output_file','w', buffering=102400) as fout:
        fout.writelines(islice(random_chars, 1000000)) # write 'n' many
    

