<h3 dir="auto">Description</h3>
<p dir="auto">My team has an airflow stack orchestrating many of our ETL jobs. Recently, we noticed a bug in one of our hourly job's code causing a large number of hourly datasets to be malformed(more than 500). Thus, after fixing the ETL job, we have the need to rerun all the impacted DAG runs to correct the data.</p>
<p dir="auto">I learnt that it's possible to batch initiate DAG reruns through the "Browse" -&gt; "Task Instances" view, where I can filter out the task instances I want to rerun through execution date and dag Id and simply clear their states. However, since we set a concurrency limit for DAG runs, after we initiate the reruns, it appears that airflow is prioritizing reruns over latest runs, so out latest DAG runs got delayed. As we have a agreement with our consumers on data processing time, we don't want the latest DAG runs to get delayed.</p>
<p dir="auto">I'm wondering if it's possible for Airflow to support a more intuitive and safe way to rerun DAG's in batch, where reruns only start if they don't impact latest runs.</p>
<h3 dir="auto">Use case/motivation</h3>
<p dir="auto">Backfill/rerun is a common use case for ETL workflows. I feel it could be valuable to make it safe and more intuitive.</p>
<h3 dir="auto">Related issues</h3>
<p dir="auto">Didn't find related issues after some simple searches. Please let me know if this is discussed somewhere else before.</p>
<h3 dir="auto">Are you willing to submit a PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
<p dir="auto"><strong>Description of the problem</strong></p>
<p dir="auto">A common workflow we have is running daily DAGs not dependent of past runs. When deployed, It need to catch up for many years before the runs are up to date and can be used by analysts.</p>
<p dir="auto">The problem is that most of times, recent runs are more used/valuable than performing ETL of execution time 20 years ago. In practice we deploy the DAG two times, one for filling recent data and one for filling since inception.</p>
<p dir="auto"><strong>Feature requested</strong></p>
<p dir="auto">A flag (<code class="notranslate">back_run=False</code>) on DAG class. If this flag is set to <code class="notranslate">True</code>, DAG runs are scheduled in reverse order.</p>
<p dir="auto">E.g. consider we're on day <code class="notranslate">10</code> and in the middle of the process a new run for day <code class="notranslate">11</code> happens:<br>
the DAG runs would be scheduled like <code class="notranslate">10 -&gt; 9 -&gt; 8 -&gt; 11 -&gt; 7 -&gt; 6 -&gt; 5</code> (assuming the run for <code class="notranslate">11</code> happened after running for day <code class="notranslate">8</code>). This flag can only be used if <code class="notranslate">depends_on_past=False</code></p>