title: Breaking down a large number of rows into smaller queries? Parallelism
body: <p>I want to create a external application which will query one table from a large Oracle database.</p>
  
  <p>The query will run daily and I am expecting to handle 30,000+ rows. </p>
  
  <p>To break down the size of these rows, I would like to create a new thread/ process for each 10,000 rows that exist. So going by the above figure it would be 3 threads to process all those rows.</p>
  
  <p>I don't want each thread to overlap each others row set so I know I will need to add a column within the table to act as a range marker, a row_position</p>
  
  <p><strong>Logic</strong></p>
  
  <pre><code>Get row_count of data set in query parameters
  Get first_row_pos
  
  While (row_count &gt; 10,000)
  {
      Create thread with 10,000 rows starting from first_row_pos
      row_count == row_count - 10,000
      first_row_pos = first_row_pos + 10,000
  }
  
  create thread for remaining rows
  all threads run their queries concurrently.
  </code></pre>
  
  <p>This is basic logic at the moment, however I do not know how feasible this is.</p>
  
  <p>Is this a good way or is there a better way?</p>
  
  <p>Can this be done through one database connection with each thread sharing or is it better to have a seperate db connection for each thread?</p>
  
  <p>Any other advice welcome?</p>
  
  <p>Note: I just realised a do while loop would be better if there is less than 10,000 rows in this case.</p>
  
  <p>Thanks</p>