title: Getting data from S3 bucket
body: <p>I have data in a S3 bucket, that is filling up day by day (data from a certain log). The bucket has different files which i read and then put in as one csv, but it is  very slow to read the bucket each time.</p>
  <p>For example:</p>
  <p>bucketxyz: file1, file2, file3
  I read the contents of file1 and extract data from this file1 into a csv then i continue with file2 and do the same and append some data from file2 to the a csv file. File 1 and File 2 are not deleted, but stay in the bucket.
  After one week, i need to do a new report using the same bucket. But now there are new files and i would reread the old ones again. So I read the whole bucket again and save it.
  The resulting CSV, i put into another bucket.</p>
  <p>What would be a efficient way to read the data from the bucket? Should i have a separate job, which puts the bucket data in a database and then read from the database?</p>