title: Ipython kernel dies unexpectedly reading large file
body: <p>I'm reading in a ~3Gb csv using pandas in an ipython notebook.  While reading the file, the notebook unexpectedly gives me an error message saying the kernel appears to have died and will restart.</p>
  
  <p>As per several "big data" workflows in python/pandas, I'm reading the files in as follows:</p>
  
  <pre><code>import pandas as pd
  tp = pd.read_csv(file_name_cleaned,chunksize,iterator=True,low_memory=False)
  df = pd.concat(tp,ignore_index=True)
  </code></pre>
  
  <p>My workflow has involved some preprocessing to remove all but alphanumeric characters and a few pieces of punctuation as follows:</p>
  
  <pre><code>with open(file_name,'r') as file1:
      with open(file_name_cleaned,'w') as file:2
          for line in file1:
                  if len(line.split(sep_string)) == num_columns:
                      line = re.sub(r'[^A-Za-z0-9|._]+','',line)
                      file2.write(line+'\n')
  </code></pre>
  
  <p>The strange thing is that if I remove the line containing re.sub(), I get a different error - "Expected 209 fileds, saw in line 22236, saw 329"  even though I've explicitly checked for the exact number of delimiters.  Visual inspection of the line and surrounding lines don't really show me much either.   </p>
  
  <p>This process has worked fine for several other files, including ones that are larger so I don't think the size of the file is the issue although I suppose it's possible that that's an oversimplification.</p>
  
  <p>I included the preprocessing because I know from experience that sometimes the data contains strange special characters, I've also gone back and forth between using encoding='utf-8' and encoding='utf-8-sig' in the read_csv() and open() statements to no real avail.</p>
  
  <p>I have several questions - does including the encoding keyword argument cause python to ignore characters outside of those character sets or does it maybe invoke some kind of conversion for those characters?  I'm not very familiar with these types of issues.  Is it possible that some kind of unexpected character could have slipped through my preprocessing and caused this?  Is there another type of issue that I haven't found that could cause this?  (I have done research but nothing has been quite right.)</p>
  
  <p>Any help would be much appreciated.</p>
  
  <p>Also, I'm using Anaconda 2.4, with Python 3.5.1, Ipython 4.0.0, and pandas 0.17.0 </p>