title: Python Strategy for Large Scale Analysis (on-the-fly or deferred)
body: <p>To analyze a large number of websites or financial data and pull out parametric data, what are the optimal strategies?</p>
  
  <p>I'm classifying the following strategies as either "on-the-fly" or "deferred".  Which is best?</p>
  
  <ol>
  <li><strong>On-the-fly</strong>: Process data on-the-fly and store parametric data into a database</li>
  <li><strong>Deferred</strong>: Store all the source data as ASCII into a file system and post process later, or with a processing-data-daemon</li>
  <li><strong>Deferred</strong>: Store all pages as a BLOB in a database to post-process later, or with a processing-data-daemon</li>
  </ol>
  
  <p>Number 1 is simplest, especially if you only have a single server.  Can #2 or #3 be more efficient with a single server, or do you only see the power with multiple servers?</p>
  
  <p>Are there any python projects that are already geared toward this kind of analysis?</p>
  
  <p>Edit: by  best, I mean fastest execution to prevent user from waiting with ease of programming as secondary</p>