title: Process different file types separately in AWS S3 (python)
body: <p>I'm looking to process files in an S3 location and re upload them to a different location depending on the file type which is specified in the key name. I want to use pagination to be able to process in batches according to accumulated file size, but I'm unsure how to do this separately for each filetype.</p>
  <p>I establish a connection to the boto3 client and relevant S3 locations earlier in the code and check that the locations and objects within them exist.</p>
  <p>My code is successfully giving me the number of each filetype in the location, but is not entering any of the <code>for</code> loops.</p>
  <p>Should I be putting the paginator code within the if statements for each filetype instead?</p>
  <pre><code>if locationExists and filesFound:
                  
      paginator = s3c.get_paginator(&quot;list_objects_v2&quot;)
      operation_parameters = {'Bucket': bucket,
                          'Prefix': path}
      page_iterator = paginator.paginate(**operation_parameters)
      lastresponse_objects = page_iterator.search(&quot;Contents[?contains(Key, `lastresponse`)][]&quot;)
      snapshot_objects = page_iterator.search(&quot;Contents[?contains(Key, `_snapshot_`)][]&quot;)
      snapshotupdate_objects = page_iterator.search(&quot;Contents[?contains(Key, `snapshotpriceupdate`)][]&quot;)                  
                  
      if lastresponse_objects:
          total_LR_files = len(list(lastresponse_objects))
          print(f'Lastresponse files: {total_LR_files}')
          for file in lastresponse_objects:
              print(file.key)
              # processing logic for files containing 'lastresponse'
                  
      if snapshot_objects:
          total_snapshot_files = len(list(snapshot_objects))
          print(f'Snapshot files: {total_snapshot_files}')
          for file in snapshot_objects:
              print(file.key)
              # processing logic for files containing '_snapshot_'
                  
      if snapshotupdate_objects:
          total_snapshotupdate_files = len(list(snapshotupdate_objects))
          print(f'Snapshot update files: {total_snapshotupdate_files}')
          for file in snapshot_objects:
              print(file.key)
              # processing logic for files containing 'snapshotpriceupdate'
  </code></pre>