title: DAG(directed acyclic graph) dynamic job scheduler
body: <p>I need to manage a large workflow of ETL tasks, which execution depends on time, data availability or an external event.  Some jobs may fail during execution of the workflow and the system should have the ability to restart a failed workflow branch without waiting for whole workflow to finish execution.</p>
  
  <p>Are there any frameworks in python that can handle this?</p>
  
  <p>I see several core functions:</p>
  
  <ul>
  <li>DAG bulding</li>
  <li>Execution of nodes (run shell cmd with wait,logging etc.)</li>
  <li>Ability to rebuild sub-graph in parent DAG during execution</li>
  <li>Ability to manual execute nodes or sub-graph while parent graph is running</li>
  <li>Suspend graph execution while waiting for external event</li>
  <li>List job queue and job details</li>
  </ul>
  
  <p>Something like <a href="http://oozie.apache.org" rel="noreferrer">Oozie</a>, but more general purpose and in python.</p>