title: Compare 2 .CSV with unknown number of columns and names
body: <p>and thanks in advance for any advice.  First-time poster here, so I'll do my best to put in all required info.  I am also quite beginner with Python, have been doing some online tutorials, and some copy/paste coding from StackOverflow, it's FrankenCoding...  So I'm probably approaching this wrong...</p>
  
  <p>I need to compare two CSV files, that will have a changing number of columns, there will only ever be 2 columns that match (for example, email_address in one file, and EMAIL in the other).  Both files will have headers, however the names of these headers may change. The file sizes may be anywhere from a few thousand lines up to +2,000,000, with potentially 100+ columns (but more likely to have a handful).  </p>
  
  <p>Output is to a third 'results.csv' file, containing all the info.  It may be a merge (all unique entries), a substract (remove entries present in one or the other) or an intersect (all entries present in both).</p>
  
  <p>I have searched here, and found a lot of good information, but all of the ones I saw had a fixed number of columns in the files.  I've tried dict and dictreader, and I know the answer is in there somewhere, but right now, I'm a bit confused.  But since I haven't made any progress in several days, and I can only devote so much time on this, I'm hoping that I can get a nudge in the right direction.</p>
  
  <p>Ideally, I want to learn how to do it myself, which means understanding how the data is 'moving around'.</p>
  
  <p>Extract of CSV files below, I didn't add more columns then (I think) necessary, the dataset I have now will match on Originalid/UID or emailaddress/email, but this may not always be the case.</p>
  
  <p>Original.csv</p>
  
  <pre><code>"originalid","emailaddress",""
  "12345678","Bob@mail.com",""
  "23456789","NORMA@EMAIL.COM",""
  "34567890","HENRY@some-mail.com",""
  "45678901","Analisa@sports.com",""
  "56789012","greta@mail.org",""
  "67890123","STEVEN@EMAIL.ORG",""
  </code></pre>
  
  <p>Compare.CSV</p>
  
  <pre><code>"email","","DATEOFINVALIDATION_WITH_TIME","OPTOUTDATE_WITH_TIME","EMAIL_USERS"
  "Bob@mail.com",,,"true"
  "NORMA@EMAIL.COM",,,"true"
  "HENRY@some-mail.com",,,"true"
  "Henrietta@AWESOME.CA",,,"true"
  "NORMAN@sports.CA",,,"true"
  "albertina@justemail.CA",,,"true"
  </code></pre>
  
  <p>Data in results.csv should be all columns from Original.CSV + all columns in Compare.csv, but not the matching one (email) :</p>
  
  <pre><code>"originalid","emailaddress","","DATEOFINVALIDATION_WITH_TIME","OPTOUTDATE_WITH_TIME","EMAIL_USERS"
  "12345678","Bob@mail.com","",,,"true"
  "23456789","NORMA@EMAIL.COM","",,,"true"
  "34567890","HENRY@some-mail.com","",,,"true"
  </code></pre>
  
  <p>Here are my results as they are now:</p>
  
  <pre><code>email,,DATEOFINVALIDATION_WITH_TIME,OPTOUTDATE_WITH_TIME,EMAIL_USERS
  Bob@mail.com,,,true,"['12345678', 'Bob@mail.com', '']"
  NORMA@EMAIL.COM,,,true,"['23456789', 'NORMA@EMAIL.COM', '']"
  HENRY@some-mail.com,,,true,"['34567890', 'HENRY@some-mail.com', '']"
  </code></pre>
  
  <p>And here's where I'm at with the code, the print statement returns matching data from the files to screen but not to file, so I'm missing something in there.<br>
  ***** And I'm not getting the headers from the original.csv file, data is coming in.</p>
  
  <pre><code>import csv
  
  def get_column_from_file(filename, column_name):
      f = open(filename, 'r')
      reader = csv.reader(f)
      headers = next(reader, None)
      i = 0
      max = (len(headers))
      while i &lt; max:
          if headers[i] == column_name:
              column_header = i
   #       print(headers[i])
          i = i + 1
      return(column_header)
  
  file_to_check = "Original.csv"
  file_console = "Compare.csv"
  
  column_to_read = get_column_from_file(file_console, 'email')
  column_to_compare = get_column_from_file(file_to_check, 'emailaddress')
  
  with open(file_console, 'r') as master:
      master_indices = dict((r[1], r) for i, r in enumerate(csv.reader(master)))
  
  with open('Compare.csv', 'r') as hosts:
      with open('results.csv', 'w', newline='') as results:
          reader = csv.reader(hosts)
          writer = csv.writer(results)
  
          writer.writerow(next(reader, []))
  
          for row in reader:
              index = master_indices.get(row[0])
              if index is not None:
                  print (row +[master_indices.get(row[0])])
                  writer.writerow(row +[master_indices.get(row[0])])
  </code></pre>
  
  <p>Thanks for your time!</p>
  
  <p>Pat</p>