title: Python jug (or other) for embarrassingly parallel jobs in cluster environment with heterogenous tasks
body: <p>I have the usual large set of dependent jobs and want to run them effectively in a PBS cluster environment.  I have been using Ruffus and am pretty happy with it, but I also want to experiment a bit with other approaches.  </p>
  
  <p>One that looks interesting in python is jug.  However, it appears that jug assumes that the jobs are homogeneous in their requirements.  I have some jobs that require 8GB RAM while others require only 100MB; some can consume all processors and some are single-threaded.  I'm aiming for being able to quickly assemble a pipeline, run it and have it "update" based on dependencies, and log reasonably so that I can see what jobs still need to be run.  Is anyone using jug or other similar system with these types of requirements?</p>