title: fastest way to split files by unique group
body: <p>What is the fastest way to split a very large file and write it to disk.</p>
  
  <p>For e.g if I have data like</p>
  
  <pre><code>chr    a_val    b_val   a_idx
  2      1355     25d     abd
  2      1785     25d     abd
  2      1825     36g     ahj
  3      1125     25d     abd
  3      1568     25d     aky
  3      2398     g67     abd
  3      1125     25d     afd
  3      1525     25d     abd
  3 ....................
  4 ..........
  4 ........
  </code></pre>
  
  <p>Where I want to split by "chr" values.</p>
  
  <p>I was thinking of applying pandas method in the following way:</p>
  
  <pre><code>my_df = pd.read_csv("my_file.txt", sep='\t')
  my_df = my_df.groupby('chr')
  
  # split the file
  for chr_, data in my_df:
      pd.Dataframe.to_csv(data, 'data_' + chr_ + '.txt', sep = '\t', header = True, index=False)
  </code></pre>
  
  <p>Pandas is very fast. But, would there be any other unix, linux or python based process method to do so in the fastest way possible.</p>
  
  <p>Thanks,</p>