Merging records from two '.CSV' files in python
Question: I have two '.csv' files in the below format: First File :

    
    
    Roll_num Class_Name
    1         ABC
    2         DEF
    5         PQR
    27        UVW
    

Second File :

    
    
    Roll_num Marks Grade
    1        75    A
    2        60    C
    27       68    B
    61       45    E
    

Now i want to add a column in the second file appending a column 'Class_Name'
from First File. The data in both the files has duplicates in it and is not
sorted.

I have written the following code that writes our required data from 2 files
into a new file.

    
    
    import csv
    
    path="xyz"
    file_read=open(path + "ClassName.CSV", "r")
    reader_ClassName = csv.reader(file_read)
    
    read_all_data=open(path + "Student.CSV", "r")
    reader_Student =csv.reader(read_all_data)
    write_all_data=open( path +"Student_Classname.CSV", "w")
    
    for line_Student in reader_Student:
            Roll_Student=line_Student[0]
            for line_ClassName in reader_ClassName:
                Roll_ClassName=line_ClassName[0]
                ClassName=line_ClassName[1]         
                if(Roll_ClassName == Roll_Student):
                    string= Roll_Student +","+ClassName  +"\n"
                    print string
                    write_all_data.write(string)
                    break  
    

Output Expected :

    
    
    Roll_num Marks Grade Class_Name
    1        75    A     ABC
    2        60    C     DEF
    27       68    B     UVW
    61       45    E     LMN
    

Output our code generates:

    
    
       Roll_num Marks Grade Class_Name
        1        75    A     ABC
        2        60    C     DEF
    

There is some issue in reading the Third line from Second inner 'for' loop. We
have hundreds of thousands of records in both the files.


Answer: I suggest to avoid `loop in loop` by reading the whole ClassName.csv file and
put into an dictionary first. I suggest the idea below

    
    
    mydict = {}
    for each_line in ClassName_csvfile:
         rollnum = get_roll_num()
         classname = get_class_name()
         mydict[rollnum]=classname 
    
    for each_line in Student_csv_file:
         rollnum = get_roll_num()
         mark = get_mark()
         grade = get_grade()
         Classname = ''
         if mydict.has_key(rollnum):
            Classname = mydict[rollnum]
         writetofile(rollnum, mark, grade, Classname)
    

Update: you can use `if rollnum in mydict:` instead of
`mydict.has_key(rollnum)` if you are using `Python 2.3+`. I am using python
2.7 and both works

P/s: Sorry for not commenting as it requires me 50 reputations

