title: Python-pandas "read_csv" is not reading the whole .TXT file
body: <p>First of all, I have found several questions with the same title/topic here and I have tried the solutions that have been suggested, but none has worked for me</p>
  <p>Here is the <strong>issue</strong>:
  I want to extract a sample of workers from a huge .txt file (&gt; 50 GB)
  I am using HPC cluster for this purpose.</p>
  <p>Every row in the data represents a worker which has many info (column variables). The idea is to a extract subsample of workers based on the first two letters in the ID variable:</p>
  <pre><code>df = pd.read_csv('path-to-my-txt-file', encoding= 'ISO-8859-1', sep = '\t', low_memory=False, error_bad_lines=False, dtype=str)
  df = df.rename(columns = {'Worker ID' : 'worker_id'})
  
  # extract subsample based on first 2 lettter in worker id
  new_df = df[df.worker_id.str.startswith('DK', na=False)]
  new_df.to_csv('DK_worker.csv', index = False)
  </code></pre>
  <p>The <strong>problem</strong> is that the resulting .CSV file has only 10-15 % of the number of rows that should be there (I have another source of information on the approximate number of rows that I should expect).</p>
  <p>I think the data has some encoding issues. I have tried something like 'utf-8', 'latin_1' .. nothing has changed.</p>
  <p>Do you see anything wrong in this code that may cause this problem? have I missed some argument?</p>
  <p><em>I am not a Python exper</em>t :)</p>
  <p>Many thanks in advance.</p>