title: Fastest way to find a file that contains a string
body: <p>I was looking for the fastest way to find a file, from a given directory, that contains a text string.</p>
  
  <p>I've come across this question: <a href="https://stackoverflow.com/questions/37078978/the-fastest-way-to-find-a-string-inside-a-file">The fastest way to find a string inside a file</a></p>
  
  <p>Most of the answers claimed that the fastest way to do this is using system grep. But no one added times so I've timed it.</p>
  
  <p>I've timed those 2 functions:</p>
  
  <pre><code>mysetup_sys = '''
  from subprocess import Popen, PIPE, STDOUT
  def find_file_containing_string(str, files_dir):
      cmd = "grep {str} -rl {dir}".format(str=str, dir=files_dir)
      proc = Popen(cmd, shell=True, stdin=None, stdout=PIPE, stderr=PIPE)
      file, err = proc.communicate()
      return file'''
  </code></pre>
  
  <p>and </p>
  
  <pre><code>mysetup_py='''
  from os import listdir
  from os.path import isfile, getmtime, join
  def find_file_containing_string(str, files_dir):
      files_paths = filter(isfile, [join(files_dir, f) for f in listdir(files_dir)])
      for file in sorted(files_paths, key=getmtime): #iterate files sorted by time
          with open(file) as fl:
              found = next((l for l in fl if str in l), None) #find first occurrence in file
          if found:
              return file
      return None'''
  </code></pre>
  
  <p>And the test:</p>
  
  <pre><code>mycode1='''find_file_containing_string(some_string_not_in_files, dir_path)'''
  mycode2='''find_file_containing_string(some_string_exists_in_one_file, dir_path)'''
  
  timeit.timeit(stmt=mycode1, number = 10000, setup=mysetup_py)
  timeit.timeit(stmt=mycode1, number = 10000, setup=mysetup_sys)
  
  timeit.timeit(stmt=mycode2, number = 10000, setup=mysetup_py)
  timeit.timeit(stmt=mycode2, number = 10000, setup=mysetup_sys)
  </code></pre>
  
  <p>I've got these results:</p>
  
  <p>String not found in files:</p>
  
  <blockquote>
    <p>Pythonic way (All files were searched): <strong>284.1225619316101</strong> </p>
    
    <p>System grep using Popen: <strong>306.4192249774933</strong></p>
  </blockquote>
  
  <p>String found in files:</p>
  
  <blockquote>
    <p>Pythonic way (Search stopped at first match, file #61 out of #87 files): <strong>211.5037169456482</strong></p>
    
    <p>Pythonic way (Search stopped at first match, file #44 out of #87
    files): <strong>177.46654987335205</strong></p>
    
    <p>System grep using Popen: <strong>317.99017000198364</strong></p>
  </blockquote>
  
  <p>The results shows that the pythonic way is ~7% faster on worst case (string not found, or found in last file) and ~33% faster on average case (string found on file #61 out of #87 files)</p>
  
  <p>The results didn't make any sense, so I tried to add --max-count 1 to grep, which means grep will stop searching within the file at first match, but it'll continue to search the other files.</p>
  
  <pre><code>cmd = "grep {str} -rlm1 {dir}".format(str=str, dir=files_dir)
  </code></pre>
  
  <blockquote>
    <p>System grep using Popen, String found in files: <strong>314.34678983688354</strong></p>
    
    <p>System grep using Popen, String not found: <strong>309.60394191741943</strong></p>
  </blockquote>
  
  <p>It seems weird that the pythonic way is that much faster, am I missing something?</p>