<h3 dir="auto">Description</h3>
<p dir="auto">A config that lets the scheduler choose tasks with the latest/oldest execution_date to run first. Or perhaps a way to dynamically assign pool slots based on the execution_date could also work.</p>
<h3 dir="auto">Use case/motivation</h3>
<p dir="auto">Sometimes the same task can have multiple task instances from different dag runs queued and occupying pool slots. The task instance from the most recent dag run could have a higher priority than the older ones. Currently there is no way to prioritize that.</p>
<h3 dir="auto">Related issues</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit a PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
<p dir="auto"><strong>Description of the problem</strong></p>
<p dir="auto">A common workflow we have is running daily DAGs not dependent of past runs. When deployed, It need to catch up for many years before the runs are up to date and can be used by analysts.</p>
<p dir="auto">The problem is that most of times, recent runs are more used/valuable than performing ETL of execution time 20 years ago. In practice we deploy the DAG two times, one for filling recent data and one for filling since inception.</p>
<p dir="auto"><strong>Feature requested</strong></p>
<p dir="auto">A flag (<code class="notranslate">back_run=False</code>) on DAG class. If this flag is set to <code class="notranslate">True</code>, DAG runs are scheduled in reverse order.</p>
<p dir="auto">E.g. consider we're on day <code class="notranslate">10</code> and in the middle of the process a new run for day <code class="notranslate">11</code> happens:<br>
the DAG runs would be scheduled like <code class="notranslate">10 -&gt; 9 -&gt; 8 -&gt; 11 -&gt; 7 -&gt; 6 -&gt; 5</code> (assuming the run for <code class="notranslate">11</code> happened after running for day <code class="notranslate">8</code>). This flag can only be used if <code class="notranslate">depends_on_past=False</code></p>