title: Column and Row manipulation Python Pandas
body: <p>This is my first program in Pandas on my own and I am trying to perform some csv manipulations by column and rows. I have a transition repository of multiple files. The transition repository keep adding file into it. I am trying to dynamically read the files and perform some operations and write to a master csv file in another folder. </p>
  
  <p>Input</p>
  
  <pre><code>1. Folder_1: `Transition_Data`  
  
  
  Test_1.csv, Test_2.csv
  
      Nos,Time,Count          Nos,Time,Count
      -------------------     ------------------
      2341,12:00:00,9865      1234,12:30:00,7865
      7352,12:00:00,8969      8435,12:30:00,7649
  
  2. Folder2: Data_repository:Master_2.csv
  
  
      Nos,00:00:00
      ------------
      1234,1000
      8435,5243
      2341,563
      7352,345
  
  3.Expected Output 
  
  Nos,00:00:00,12:00:00,12:30:00
  ----------------------------------
  1234,1000,0,6865
  8435,5243,0,2406
  2341,563,9302,0
  7352,345,8624,0
  </code></pre>
  
  <p>Read <code>Nos</code> column from transition_data files and check where <code>Nos</code> is located in <code>Master_2.csv</code> The create a new column with <code>Time</code> as new header every time and subtract the <code>col[2]</code> of Transition_data files with <code>col[1]</code> of <code>Master_2.csv</code> fill the new value in newly created column if data gaps be filled with <code>0</code>. I did try with few examples around but I messed up. </p>
  
  <p>Program Updates as explained below and now have problem routing the logic with file read and write</p>
  
  <pre><code>    import pandas as pd
      import os
      import numpy as np
      import glob
  
  path_1 = '/Transition_Data/'
  path_2 = 'Data_repository/Master_2.csv'
  
  df_1 = pd.DataFrame(dict(Nos=Nos, Time=Time, Count=Count))
  
  pivot = pd.pivot_table(path_1, '/.*CSV, index='Nos', columns='Time', values='Count')
  
  df_master = pd.DataFrame('Master_2.csv', {'Nos':, '00:00:00':}).set_index('Nos')
  
  result = df_master.join(pivot, how='inner')
  
  result[result.columns[1:]] = result[result.columns[1:]].sub(result[result.columns[0]], axis=0)
  
  result.fillna(0)
  </code></pre>
  
  <p>I tried the above program and got the following error </p>
  
  <pre><code>Traceback (most recent call last):
    File "read_test.py", line 19, in &lt;module&gt;
      df = pd.read_csv(filename, header='Count')
    File "/usr/lib/python2.7/dist-packages/pandas/io/parsers.py", line 420, in parser_f
      return _read(filepath_or_buffer, kwds)
    File "/usr/lib/python2.7/dist-packages/pandas/io/parsers.py", line 218, in _read
      parser = TextFileReader(filepath_or_buffer, **kwds)
    File "/usr/lib/python2.7/dist-packages/pandas/io/parsers.py", line 502, in __init__
      self._make_engine(self.engine)
    File "/usr/lib/python2.7/dist-packages/pandas/io/parsers.py", line 610, in _make_engine
      self._engine = CParserWrapper(self.f, **self.options)
    File "/usr/lib/python2.7/dist-packages/pandas/io/parsers.py", line 972, in __init__
      self._reader = _parser.TextReader(src, **kwds)
    File "parser.pyx", line 476, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:4538)
  TypeError: an integer is required
  </code></pre>