title: Combining CSV Files with additional columns - and aligning columns
body: <p>We have been receiving files from our counterparty for the past 3 years on a daily basis. So that means we have well over 1000 files now. Each of them have between 5000 to 15000 rows depending on the day.</p>
  <p>I was trying to combine it in Python using some googling and research in Visual Studio Code. To test things out I took only the Files from the last day of each month. A total of 33 files.</p>
  <p>The files are like this</p>
  <pre><code>File 1:
  
  Header_1  Header_2      Header 3 
  0         2             1
  2         3             2 
  4                       3  
  
  
  File 2:     
  
  Header_1   Header_4      Header_3  Header_2
  6          4             3         1
  8          5             4 
  10
  
  Desired Output
  Header_1   Header_2   Header_3   Header_4 File_Name
  0          2          1                   File 1
  2          3          2                   File 1
  4                     3                   File 1
  6          1          3          4        File 2
  8                     4          5        File 2
  10
  </code></pre>
  <p>The code I used to try this is:</p>
  <pre><code>import os
  import pandas as pd
  import glob
  
  #set working directory
  os.chdir(&quot;/filepath/&quot;)
  
  globbed_files = glob.glob(&quot;*.csv&quot;) #creates a list of all csv files
  print(globbed_files)
  data = [] # pd.concat takes a list of dataframes as an agrument
  for csv in globbed_files:
      frame = pd.read_csv(csv)
      data.append(frame)
      print (frame) #to check while running whether the frame was correct
  
  
  bigframe = pd.concat(data, ignore_index=True, keys=globbed_files) 
  bigframe.to_csv(&quot;output.csv&quot;)
  </code></pre>
  <p>I can give up the File Names if needed and the empty cells can be NaN or just empty that's fine. But right now my header's aren't lining up and I will get completely mismatched columns.</p>