title: How to write Rake task to import data to Rails app?
body: <p><strong>Goal:</strong> Using a CRON task (or other scheduled event) to update database with nightly export of data from an existing system.</p>
  
  <p>All data is created/updated/deleted in an existing system. The website does no directly integrate with this system, so the rails app simply needs to reflect the updates that appear in the data export.</p>
  
  <p>I have a <code>.txt</code> file of ~5,000 products that looks like this:</p>
  
  <pre><code>"1234":"product name":"attr 1":"attr 2":"ABC Manufacturing":"2222"
  "A134":"another product":"attr 1":"attr 2":"Foobar World":"2447"
  ...
  </code></pre>
  
  <p>All values are strings enclosed in double quotes (<code>"</code>) that are separated by colons (<code>:</code>)</p>
  
  <p>Fields are:</p>
  
  <ul>
  <li><code>id</code>: unique id; alphanumeric</li>
  <li><code>name</code>: product name; any character</li>
  <li>attribute columns: strings; any character (e.g., size, weight, color, dimension)</li>
  <li><code>vendor_name</code>: string; any character</li>
  <li><code>vendor_id</code>: unique vendor id; numeric</li>
  </ul>
  
  <p>Vendor information is not normalized in the current system.</p>
  
  <p>What are best practices here? Is it okay to delete the products and vendors tables and rewrite with the new data on every cycle? Or is it better to only add new rows and update existing ones?</p>
  
  <p><strong>Notes:</strong></p>
  
  <ol>
  <li>This data will be used to generate <code>Orders</code> that will persist through nightly database imports. <code>OrderItems</code> will need to be connected to the product ids that are specified in the data file, so we can't rely on an auto-incrementing primary key to be the same for each import; the unique alphanumeric id will need to be used to join <code>products</code> to <code>order_items</code>.</li>
  <li>Ideally, I'd like the importer to normalize the Vendor data</li>
  <li>I cannot use vanilla SQL statements, so I imagine I'll need to write a <code>rake</code> task in order to use <code>Product.create(...)</code> and <code>Vendor.create(...)</code> style syntax.</li>
  <li>This will be implemented on EngineYard</li>
  </ol>