Python function to hash a python function I'm working on a sort of distributed build system. The system allows execution of snippets of scripts as build steps. I need to be able to hash these snippets of code in a way that comments and doc strings don't affect the hash. I'm getting part way there by using the ast module to parse the code, then doing an ast.dump and hashing the resulting string. The natural next step for me is to clear the first Expr(str()) node in the body of every FunctionDef node.

This is the best solution I've found so far.

Is there a better way to solve this problem?

import ast
import hashlib
import inspect

def _remove_docstring(node):
    '''
    Removes all the doc strings in a FunctionDef or ClassDef as node.
    Arguments:
        node (ast.FunctionDef or ast.ClassDef): The node whose docstrings to
            remove.
    '''
    if not (isinstance(node, ast.FunctionDef) or
            isinstance(node, ast.ClassDef)):
        return

    if len(node.body) != 0:
        docstr = node.body[0]
        if isinstance(docstr, ast.Expr) and isinstance(docstr.value, ast.Str):
            node.body.pop(0)

#-------------------------------------------------------------------------------
def hash_function(func):
    '''
    Produces a hash for the code in the given function.
    Arguments:
        func (types.FunctionObject): The function to produce a hash for
    '''
    func_str = inspect.getsource(func)
    module = ast.parse(func_str)

    assert len(module.body) == 1 and isinstance(module.body[0], ast.FunctionDef)

    # Clear function name so it doesn't affect the hash
    func_node = module.body[0]
    func_node.name = "" 

    # Clear all the doc strings
    for node in ast.walk(module):
        _remove_docstring(node)

    # Convert the ast to a string for hashing
    ast_str = ast.dump(module, annotate_fields=False)

    # Produce the hash
    fhash = hashlib.sha256(ast_str)
    result = fhash.hexdigest()
    return result

#-------------------------------------------------------------------------------
# Function 1
def test(blah):
    'This is a test'
    class Test(object):
        '''
        My test class
        '''
    print blah
    def sub_function(foo):
        '''arg'''
print hash_function(test)

#-------------------------------------------------------------------------------
# Function 2
def test2(blah):
    'This is a test'
    class Test(object):
        '''
        My test class
        '''
    print blah
    def sub_function(foo):
        '''arg meh'''
print hash_function(test2)
Looking at the source code for ast.get_docstring your code more or less does what it does. Here is their code for reference:

def get_docstring(node, clean=True):
    """
    Return the docstring for the given node or None if no docstring can
    be found.  If the node provided does not have docstrings a TypeError
    will be raised.
    """
    if not isinstance(node, (AsyncFunctionDef, FunctionDef, ClassDef, Module)):
        raise TypeError("%r can't have docstrings" % node.__class__.__name__)
    if not(node.body and isinstance(node.body[0], Expr)):
        return
    node = node.body[0].value
    if isinstance(node, Str):
        text = node.s
    elif isinstance(node, Constant) and isinstance(node.value, str):
        text = node.value
    else:
        return
    if clean:
        import inspect
        text = inspect.cleandoc(text)
return text


A couple of the things they mention are only a concern for Python 3+ like AsyncFunctionDef. Also, it seems Constant is a grammar rule introduced in Python 3+. (However, you may want to account for this in the future.)

You seem to be also removing docstrings from ClassDef, I am not one hundred percent sure, but you may also want to add Module as well.

There is also a ast.NodeTransformer. However, I find it rather weird stylistically compared to your current method of looping. So... I'm not confident in making any comments about whether you should be using it or not. Nevertheless, I'm putting it out there so you are aware in case you weren't.