Feeding a List of Lists into a function
<p>I have a multi-dimensional array that I am trying to feed into <code>difflib.get_close_matches()</code>.</p>

<p>My array looks like this: <code>array[(ORIGINAL, FILTERED)]</code>. <code>ORIGINAL</code> is a string, and <code>FILTERED</code> is the <code>ORIGINAL</code> string with common words filtered out. </p>

<p>I currently have a new array being created with only the <code>FILTERED</code> words being fed into <code>difflib.get_close_matches()</code>. I then try to match the result from <code>difflib</code> to the <code>array[(ORIGINAL, FILTERED)]</code>. My problem is that often times I have two or more <code>FILTERED</code> words that are equivalent and therefore they can't be matched using this method.</p>

<p>Is there a way where I can feed the entire <code>array[(ORIGINAL,FILTERED)]</code> into <code>difflib</code>, but have it only look at the <code>FILTERED</code> part (while still returning the <code>[(ORIGINAL,FILTERED)]</code>?)</p>

<p>Thanks in advance!</p>

<pre><code>import  time
import  csv
import  difflib
import  sys
import  os.path
import  datetime

### Filters out common  words   in  an  attempt to  get better      results ###
def ignoredWords (word):
    filtered = word.lower()
    #Common Full Words
## Majority of filters were edited out
    #Common Abbreviations
    if "univ" in filtered:
        filtered = filtered.replace("univ","")
    #Special Characters
    if "  " in filtered: #Two White Spaces
        filtered = filtered.replace("  "," ")
    if "-" in filtered:
        filtered = filtered.replace("-"," ")
    if "\'" in filtered:
        filtered = filtered.replace("\'"," ")
    if " &amp; " in filtered:
        filtered = filtered.replace(" &amp;","")
    if "(\"" in filtered:
        filtered = filtered.replace("(\"","")
    if "\")" in filtered:
        filtered = filtered.replace("\")","")
    if "\t" in filtered:
        filtered = filtered.replace("\t"," ")
    return  filtered

### Takes in a list, then outputs a 2D list. array[Original, Filtered] ###
### For XXX: array[Original, Filtered, Account Number, Code] ###
def create2DArray (list):
    array = []
    for item in list:
        clean = ignoredWords(item[2])
        entry = (item[2].lower(), clean, item[0],item[1])
        array.append(entry)
    return array

def main(argv):
    if(len(argv) &lt; 3):
        print "Not enough parameters. Please enter two file names"
        sys.exit(2)
    elif (not os.path.isfile(argv[1])):
        print "%s is not found" %(argv[1])
        sys.exit(2)
    elif (not os.path.isfile(argv[2])):
        print "%s is not found" %(argv[2])
        sys.exit(2)
    #Recode File ----- Not yet implemented
#       if(len(argv) == 4):
#       if(not os.path.isfile(argv[3])):
#           print "%s is not found" %(argv[3])
#           sys.exit(2)
#           
#       recode = open(argv[1], 'r')
#       try:
#           setRecode = c.readlines()
#       finally:
#           recode.close()
#           setRecode.sort()
#           print setRecode[0]
    #Measure execution time
    t0 = time.time()

    cReader = csv.reader(open(argv[1], 'rb'), delimiter='|')
    try:
        setC = []
        for row in cReader:
            setC.append(row)
    finally:
        setC.sort()

    aReader = csv.reader(open(argv[2], 'rb'), delimiter='|')
    try:
        setA = []
        for row in aReader:
            setA.append(row)
    finally:
        setA.sort()

    #Put Set A and Set C into their own 2 dimmensional arrays.array[Original Word]    [Cleaned Up Word]
    arrayC = create2DArray(setC)
    arrayA = create2DArray(setA)

    #Create clean list versions for use with difflib
    cleanListC = []
    for item in arrayC:
        cleanListC.append(item[1])

    cleanListA = []
    for item in arrayA:
        cleanListA.append(item[1])

    ############OUTPUT FILENAME############
    fMatch75 = open("Match75.csv", 'w')
    Match75 = csv.writer(fMatch75, dialect='excel')
    try:
        header = "Fuzzy Matching Report. Generated: "
        header += str(datetime.date.today())
        Match75.writerow([header])
        Match75.writerow(['C','A','C Cleaned','A Cleaned','C Account', 'C Group','A Account', 'A Group', 'Filtered Ratio %','Unfiltered Ratio %','Average Ratio %'])
        for item in cleanListC:
            match = difflib.get_close_matches(item,cleanListA,1,0.75)

            if len(match) &gt; 0:
                filteredratio = difflib.SequenceMatcher(None,item,match[0]).ratio()
                strfilteredratio = '%.2f' % (filteredratio*100)
                found = 0
                for group in arrayA:
                    if match[0] == group[1]:
                        origA = group[0]
                        acode = group[3]
                        aaccount = group[2]
                        found = found + 1
                for group in arrayC:
                    if item == group[1]:
                        origC = group[0]
                        ccode = group[3]
                        caccount = group[2]
                        found = found + 2
                if found == 3:
                    unfilteredratio = difflib.SequenceMatcher(None,origC,origA).ratio()
                    strunfilteredratio = '%.2f' % (unfilteredratio*100)
                    averageratio = (filteredratio+unfilteredratio)/2
                    straverageratio = '%.2f' % (averageratio*100)

                    row = [origC.rstrip(),origA.rstrip(),item.rstrip(),match[0].rstrip(),caccount,ccode,aaccount,acode,strfilteredratio,strunfilteredratio,straverageratio]
                    Match75.writerow(row)
                #These Else Ifs are for debugging. If NULL is found anywhere in the CSV, then an error has occurred
                elif found == 2:
                    row = [origC.rstrip(),"NULL",item.rstrip(),match[0].rstrip(),caccount,ccode,"NULL","NULL",strfilteredratio,"NULL","NULL"]
                    Match75.writerow(row)
                elif found == 1:
                    row = ["NULL",origA.rstrip(),item.rstrip(),match[0].rstrip(),"NULL","NULL",aaccount,acode,strfilteredratio,"NULL","NULL"]
                    Match75.writerow(row)
            else:
                    row = ["NULL","NULL",item.rstrip(),match[0].rstrip(),"NULL","NULL","NULL","NULL",strfilteredratio,"NULL","NULL"]
                    Match75.writerow(row)

    finally:
        Match75.writerow(["A Proprietary and Confidential. Do Not Distribute"])
        fMatch75.close()

    print (time.time()-t0,"seconds")

if __name__ == "__main__":
    main(argv=sys.argv)
</code></pre>

<p><strong>What I am trying to achieve:</strong></p>

<ol>
<li>Read input files</li>
<li>Filter out common words from names so that the fuzzy matching ('difflib.get_close_matches()') will return more accurate results</li>
<li>Compare names from FileA to names in FileB to find which one is most likely a match.</li>
<li>Print out the original (un-filtered) names and the match percentage.</li>
</ol>

<p><strong>Why this is difficult</strong></p>

<p>The naming conventions used in the two input files vary significantly. Some of the names are partially abbreviated (EX: File A: Acme Company; File B: Acme Co). Since the naming conventions aren't consistent, I can't do 'FileA.intersect(FileB)' which would have been the ideal way.</p>

<p><strong>Where the modification should occur</strong></p>

<pre><code>for item in cleanListC:
    match = difflib.get_close_matches(item,cleanListA,1,0.75)
</code></pre>

<p>CleanListA is created by:</p>

<pre><code>cleanListA = []
    for item in arrayA:
        cleanListA.append(item[1])
</code></pre>

<p>Thus losing the <code>(ORIGINAL,FILTERED)</code> pairing.</p>

<p><strong>End Goal</strong></p>

<p>I would like to feed arrayA into <code>difflib.get_close_matches()</code> instead of cleanListA to preserve the <code>(ORIGINAL,FILTERED)</code> pairing. <code>difflib.get_close_matches()</code> would only look at the 'FILTERED' part of the pairing when determining the close matches, but return the entire pairing.</p>
