title: Paperclip, Delayed Job, S3, Heroku - design for delayed processing of sensitive uploaded files: db or s3?
body: <p>I need feedback on the design for uploading and delayed processing of a file using heroku, paperclip, delayed job and, if necessary, s3. Parts of it have been discussed in other places but I couldn't find a complete discussion anywhere.</p>
  
  <p>Task description:</p>
  
  <ol>
  <li>Upload file (using paperclip to s3/db on heroku). File needs to be private as it contains sensitive data.</li>
  <li>Queue file for processing (delayed job)</li>
  <li>Job gets run in queue </li>
  <li>File is retrieved (from s3/db), and processing is completed</li>
  <li>File is deleted (from s3/db)</li>
  </ol>
  
  <p>Since I am using delayed job, I have to decide between storing the file in the database or on s3. I am assuming that storing the file on the web server is out of the question as I am using heroku and delayed job. Uploading files to s3 takes a long time. But, storing files in db is more expensive. Ideally, we would want the processing to finish as quickly as possible.</p>
  
  <p>What is the more common design pattern? Store files on s3? Store files in db? Any particular recommended gems used to retrieve and process files stored in s3 (aws-s3? s3?)?</p>