title: How to extract text from log files in Python
body: <p>I have found some code online and am trying to use it to extract any line of text that has the word error in it from multiple log files. I am able to read the files in the path and output them on the screen but cannot actually extract the lines that have 'error' in them.</p>
  <p>The code that I have put together from online searches is below. When running the code, all of the .log files are shown and the output file is created, but lists the actual file name and not the lines I need. Out of the nine .log files I have, I should receive 5 lines containing errors. I did find information for extracting information from PDFs but from what I see online it is completely different than what I am trying to do.</p>
  <p>About 4-5 times a month, we manually search anywhere from 10-50 .log files looking for errors and I want to be able to dump all the logs in a folder and run a python script to find the errors very quickly. Am I going about this the wrong way?</p>
  <pre><code>import os
  # defining location of parent folder
  BASE_DIRECTORY = '/pending/PHS'
  output_file = open('/pending/PHS/output.txt', 'w')
  output = {}
  file_list = []
  
  # scanning through sub folders
  for (dirpath, dirnames, filenames) in os.walk(BASE_DIRECTORY):
      for f in filenames:
          if 'log' in str(f):
              e = os.path.join(str(dirpath), str(f))
              file_list.append(e)
  
  for f in file_list:
      print(f)
  txtfile = open(f, 'r')
  output[f] = []
  
  for line in txtfile:
      if 'error' in line:
          output[f].append(line)
  tabs = []
  
  for tab in output:
      tabs.append(tab)
  
  tabs.sort()
  
  for tab in tabs:
      output_file.write(tab + '\n')
      output_file.write('\n')
  
  for row in output[tab]:
      output_file.write(row + '')
      output_file.write('\n')
      output_file.write('----------------------------------------------------------\n')
  </code></pre>