title: Move data from oracle to HDFS, process and move to Teradata from HDFS
body: <p>My requirement is to</p>
  
  <blockquote>
    <ol>
    <li>Move data from Oracle to HDFS</li>
    <li>Process the data on HDFS</li>
    <li>Move processed data to Teradata.</li>
    </ol>
  </blockquote>
  
  <p>It is also required to do this entire processing every 15 minutes. The volume of source data may be close to 50 GB and the processed data also may be the same.</p>
  
  <p>After searching a lot on the internet, i found that</p>
  
  <blockquote>
    <ol>
    <li>ORAOOP to move data from Oracle to HDFS (Have the code withing the shell script and schedule it to run at the required interval).</li>
    <li>Do large scale processing either by Custom MapReduce or Hive or PIG.</li>
    <li>SQOOP - Teradata Connector to move data from HDFS to Teradata (again have a shell script with the code and then schedule it).</li>
    </ol>
  </blockquote>
  
  <p>Is this the right option in the first place and is this feasible for the required time period (Please note that this is not the daily batch or so)?</p>
  
  <p>Other options that i found are the following</p>
  
  <blockquote>
    <ol>
    <li>STORM (for real time data processing). But i am not able to find the oracle Spout or Teradata bolt out of the box.</li>
    <li>Any open source ETL tools like Talend or Pentaho.</li>
    </ol>
  </blockquote>
  
  <p>Please share your thoughts on these options as well and any other possibilities.</p>