<s>[INST] Get MD5 hash of big files in Python I have used hashlib (which replaces md5 in Python 2.6/3.0) and it worked fine if I opened a file and put its content in hashlib.md5() function.
The problem is with very big files that their sizes could exceed RAM size.
How to get the MD5 hash of a file without loading the whole file to memory?
 [/INST] You need to read the file in chunks of suitable size:
def md5_for_file(f, block_size=2**20):
    md5 = hashlib.md5()
    while True:
        data = f.read(block_size)
        if not data:
            break
        md5.update(data)
    return md5.digest()

NOTE: Make sure you open your file with the 'rb' to the open - otherwise you will get the wrong result.
So to do the whole lot in one method - use something like: 
def generate_file_md5(rootdir, filename, blocksize=2**20):
    m = hashlib.md5()
    with open( os.path.join(rootdir, filename) , "rb" ) as f:
        while True:
            buf = f.read(blocksize)
            if not buf:
                break
            m.update( buf )
    return m.hexdigest()

The update above was based on the comments provided by Frerich Raabe - and I tested this and found it to be correct on my Python 2.7.2 windows installation
I cross-checked the results using the 'jacksum' tool.
jacksum -a md5 <filename>

http://www.jonelo.de/java/jacksum/
 </s>