processes and threading finallyself work_queue task_done(the differences are that we have more shared data to keep track of and we call our custom process(function with different arguments we don' have to worry about the queues since they ensure that accesses are serializedbut for other data itemsin this case the md _from_filename dictionarywe must handle the serialization ourselves by providing lock we have made the lock class attribute because we want every worker instance to use the same lock so that if one instance holds the lockall the other instances are blocked if they try to acquire it we will review the process(function in two parts def process(selfsizefilenames)md collections defaultdict(setfor filename in filenameswith self md _lockmd self md _from_filename get(filenamenoneif md is not nonemd [md add(filenameelsetrymd hashlib md (with open(filename"rb"as fhmd update(fh read()md md digest(md [md add(filenamewith self md _lockself md _from_filename[filenamemd except environmenterrorcontinue we start out with an empty default dictionary where each key is to be an md digest value and where each value is to be set of the filenames of the files that have the corresponding md value we then iterate over all the filesand for each one we retrieve its md if we have already calculated itand calculate it otherwise context managers whether we access the md _from_filename dictionary to read it or to write to itwe put the access in the context of lock instances of the threading lock(class are context managers that acquire the lock on entry and release the lock on exit the with statements will block if another thread has the md _lockuntil the lock is released for the first with statement when we acquire the lock we get the md from the dictionary (or none if it isn' thereif the md is none we must compute itin which case we store it in the md _from_filename dictionary to avoid performing the computation more than once per file