title: Workload Distribution / Parallel Execution in JAVA
body: <p>I have a situation here where I need to distribute work over to multiple JAVA processes running in different JVMs, probably different machines.</p>
  
  <p>Lets say I have a table with records 1 to 1000. I am looking for work to be collected and distributed is sets of 10. Lets say records 1-10 to workerOne. Then records 11-20 to workerThree. And so on and so forth. Needless to say workerOne never does the work of workerTwo unless and until workerTwo couldnt do it.</p>
  
  <p>This example was purely based on database but could be extended to any system, I believe be it File processing, email processing and so forth.</p>
  
  <p>I have a small feeling that the immediate response would be to go for a Master/Worker approach. However here we are talking about different JVMs. Even if one JVM were to come down the other JVM should just keep doing its work.</p>
  
  <p>Now the million dollar question would be: Are there any good frameworks(production ready) that would give me facility to do this. Even if there are concrete implementations of specific needs like Database records, File processing, Email processing and their likes.</p>
  
  <p>I have seen the Java Parallel Execution Framework, but am not sure if it can be used for different JVMs and if one were to come down would the other keep going.I believe Workers could be on multiple JVMs, but what about the Master?</p>
  
  <p>More Info 1: Hadoop would be a problem because of the JDK 1.6 requirement. Thats bit too much.</p>
  
  <p>Thanks,
  Franklin</p>