I have a large function that performs a search over a data set given a set of parameters. This function goes through multiple databases which all hold data that would comprise one final entry from the search.def perform(self):    #Go through each year and each DB in that year    for year, yearInfo in data[preprocessDBInfo].items():        termsToCases = {}        for dbName, dbInfo in yearInfo[dbs].items():            #Relevant to search?            relevantTerms = self.search.ofDB(dbName)            if len(relevantTerms) == 0:                continue #Not a relevant database            #Open the database and get cases            fp = os.path.join(prefs[rootPath], dbInfo[filePath])            caseDB = caseDB(fp)            cases = caseDB.getCases(stubs=True,search=relevantTerms)            termsToCases.update(cases)        self.foundCases = self.foundCases.union(self.resolve(termsToCases))Anyone who wants to call this function would have to wait for the entire search to complete which can take a couple of minutes depending on the search complexity.Now I need to move this task to a separate thread of execution and be able to periodically check it and more drastically pause or kill it. In order to do this I need to break it up so that perform only does small work units so the caller of it has much more control over execution. I have come up with the following to solve thisdef performGenerator(self):    #Go through each year and each DB in that year    for year, yearInfo in data[preprocessDBInfo].items():        for dbName, dbInfo in yearInfo[dbs].items():            #Relevant to search?            relevantTerms = self.search.ofDB(dbName)            if len(relevantTerms) == 0:                continue #Not a relevant database            #Open the database and get cases            fp = os.path.join(prefs[rootPath], dbInfo[filePath])            caseDB = caseDB(fp)            cases = caseDB.getCases(stubs=True,search=relevantTerms)            yield casesIs there any sort of name for a function like this, a function that originally was large and autonomous but now only does tiny work units? (disregarding the fact that it's a generator in Python)Is there any sort of pattern or methodology for breaking up a large subtask list this? Python's generator interface seems like it would be the best fit for my needs but I don't know what I would do if I didn't have such a high level interface (such as if I were programming in C)
Aggregating search results from multiple databases, in smaller work units
python;database;generator