perform fuzzy matching on a large set of dictionary keys Any suggestions on improving the speed of below code. My dictionary is rather large in fold of * 100000 and expected to grow larger every day.
I am open to completely new ideas or approaches to do the below operation.
from pyxdameraulevenshtein import damerau_levenshtein_distance, normalized_damerau_levenshtein_distance
name = 'john doe'

word_dict = { 
              'john' : ['ID1','ID2', 'AB2' ,'AS1']
             ,'doe' : ['ID1','ID4', 'AB2' ,'AS6']
             ,'jahn' : ['ID3','ID2', 'AB2' ,'AS5']
             }
# perform iteration on dictionary keys . compute and filter if damerau edit distance is less than 2 
all_matches = []
for nWord in set(name.split()):
    match = []
    match += [(word,list(word_dict[word]), int((1-normalized_damerau_levenshtein_distance(nWord,word))* 100))\
    for word in word_dict if damerau_levenshtein_distance(nWord,word) in [0,1,2]]
    all_matches.append(match)
Take a look at BKTrees. Here is a less technical description. There is a Python library on PyPI: pybktree.
You create the BKTree once (once a day?) from the keys in the dictionary. Then run all the queries against the BKTree.
It would look something like this untested code:
import pybktree
from pyxdameraulevenshtein import damerau_levenshtein_distance

tree = pybktree.BKTree(damerau_levenshtein_distance, word_dict.keys())

all_matches = []

for target in set(name.split()):
    all_matches.extend(tree.find(target, MAX_DISTANCE))]

all_matches would be a list of (distance, word) tuples.