title: Why re function take so much memory
body: <p>I loaded a data file(34 million line sentence) which took 4Gigabyte memory on my laptop.</p>
  
  <p>While I was doing preprocessing, the memory increased 1.5G after 2 million sentences get processed. </p>
  
  <pre><code>count = 0
  for line in lines:
      lines[count] = re.findall(r"[\w']+|[().,:!?;'$&amp;]", line)
      count += 1
      if count % 100000 == 0:
          print(count)
          gc.collect()
  </code></pre>
  
  <p>Can somebody explain why and how to optimize it? </p>