title: Improve the speed of for loop for readline of very big files
body: <p>I am trying to improve the speed of my for loop for reading lines from very big files. I have two files, I am taking information from first file line by line in a <code>for</code> loop and matching these each to lines from a second file through an <code>if</code> statement. Since both the files have millions of lines, this is taking too long.</p>
  
  <p>I am posting my code here; How can I improve the loop statement to increase the speed of execution?</p>
  
  <pre><code>#!/usr/bin/python
  
  #open file1
  f1 = open("../../Reapeat_analysis/FIMO/fimo_out/fimo.gff",'r')
  #open file2
  f2 = open("../BS_Forward.fastq_bismark_pe.CX_report.txt",'r')
  
  f1.seek(0)
  f2.seek(0)
  
  #open and save output
  fOut = open("output_sample_CG+.txt",'w')
  
  #Reading file1 lines in for loop
  for line1 in f1:
      line1 = line1.split('\t')
      s1 = int(line1[3])
      s2 = int(line1[4])
      s0 = str(line1[0])
      count = 0
      percent = 0
      lt = []
  
      #Reading file2 lines for each file1 line
      for line2 in f2:
          line2 = line2.split("\t")
  
          #Matching desired condition
          if (s0 == str(line2[0])) and (s1 &lt;= int(line2[1]) &lt;= s2) and (str(line2[5])=="CG") and (str(line2[2])=="+"):
              lt.append(line2)
              count = count + 1
  
      #saving each matched conditions
      fOut.write(str(s1) + "-" + str(s2) + ":" + str(s0) + "\t" + str(count) + "\t" + str(lt))
      f2.seek(0)
  fOut.close()
  </code></pre>
  
  <p>Between 0 and 100 lines of the <code>f2</code> file match the <code>(str(line2[5])=="CG") and (str(line2[2])=="+")</code> filter.</p>