title: python deleting older files that match an unknown pattern (tricky)
body: <p>My server is filling up and I need to automate file deletion. Files generally get added to my server daily but sometimes there are pauses making them bi-weekly and or monthly. They stop coming in for months and then start again, it's unpredictable.</p>
  
  <p>My script needs to delete files that are older than 30 days <strong>but always keep the newest 5 files of any file pattern it finds</strong>. This is the tricky part. </p>
  
  <p>The only predictable thing/pattern about the files is the files always contain a yyyymmddhhmmss timestamp somewhere and some pattern that repeats,
  other parts of the file names are not always predictable. If a file has no timestamp I don't want to delete it.</p>
  
  <p>An example directory my have something like this</p>
  
  <p>20121118011335_team1-pathway_Truck_Report_Data_10342532.zip<br>
  20121119011335_team1-pathway_Truck_Report_Data_102345234.zip<br>
  20121120011335_team1-pathway_Truck_Report_Data_10642224.zip<br>
  20121121011335_team1-pathway_Truck_Report_Data_133464.zip<br>
  20121122011335_team1-pathway_Truck_Report_Data_126434344.zip<br>
  20121123011335_team1-pathway_Truck_Report_Data_12444656.zip<br>
  20121124011335_team1-pathway_Truck_Report_Data_1624444.zip<br>
  20121125011335_team1-pathway_Truck_Report_Data_3464433.zip<br>
  randomefilewithnodate.zip<br>
  20121119011335_team2-Paper_Size_Report_336655.zip<br>
  20121120011335_team2-Paper_Size_Report_336677.zip<br>
  20121121011335_team2-Paper_Size_Report_338877.zip<br>
  20121122011335_team2-Paper_Size_Report_226688.zip<br>
  20121123011335_team2-Paper_Size_Report_776688.zip<br>
  20121124011335_team2-Paper_Size_Report_223355.zip<br>
  20121125011335_team2-Paper_Size_Report_111111.zip</p>
  
  <p>In such a case my script should only be deleting the oldest 3 files of that first pattern
  20121118011335_team1-pathway_Truck_Report_Data_10342532.zip<br>
  20121119011335_team1-pathway_Truck_Report_Data_102345234.zip<br>
  20121120011335_team1-pathway_Truck_Report_Data_10642224.zip  </p>
  
  <p>and the oldest 2 files of the second pattern<br>
  20121119011335_team2-Paper_Size_Report_336655.zip<br>
  20121120011335_team2-Paper_Size_Report_336677.zip   </p>
  
  <p><em>this way it keeps 5 newest files and doesn't touch the file with no date</em></p>
  
  <p>My problem is I have no way of knowing what exactly will follow yyyymmddhhmmss_
  I only know it will be various iterations of yyyymmddhhmmss_something_consistent_random_random or yyyymmddhhmmss_something_consistent_something_consistent_random_random.xyz</p>
  
  <p>So far I've come up with the regex to match if a time stamp exists, but I can't think of how to get my script to be smart enough to detect the rest of the file's pattern and keep 5 days of a pattern. </p>
  
  <p>Any ideas are welcome! The script below is not perfect, <em>I can fix the small mistakes.</em> </p>
  
  <p><strong>I really need help with the keeping 5 newest files part mainly</strong></p>
  
  <p>Bonus question is the epoc time part. </p>
  
  <pre><code>def myCleansingMethod(self, client)
  
      # Get rid of things older than 30 days
      # 30 days has this many seconds 30 * 24 * 60 * 60
      numberOfSeconds = 2592000
      # establish what the epoc time of the oldest file I want to keep is
      oldestFileThatIWantToKeep = time.time() - numberOfSeconds
      #establish my working directory
      workingDirectory = "/home/files/%s" % (client)
      try:
          files = os.listdir(workingDirectory)
          except:
          print "Could not find directory"
          return
  
          files.sort()
          for file in files:
              # define Full File Name (path + file)
              fullFileName = "%s/%s" % (workingDirectory, file)
              # make sure the file contains yyyymmddhhmmss
              match = re.search(r'[0-9]{4}(1[0-2]|0[1-9])(3[01]|[12][0-9]|0[1-9])([01]\d|2[0123])([0-5]\d){2}', file)
              if match:
                  #get what was matched in the RegEx
                  fileTime = match.group()
                  #convert fileTime to Epoc time
                  fileTimeToEpoc = (fileTime + NOT SURE HOW TO DO THIS PART YET)
  
                  if fileTimeToEpoc &lt; oldestFileThatIWantToKeep AND (CODE THAT MAKES SURE   THERE ARE AT LEAST 5 FILES OF THE SAME PATTERN PRESENT) :
                  print "Delete file: %s\t%s" % (fileTimeToEpoc, fullFileName)
                  command = "rm -Rf %s" % fullFileName
                  print command
                  os.system (command)
                  else:
                  pass  
              else:
              pass
  </code></pre>