<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.3.3</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">I have multiple dags running regularly. Dags fail randomly for no reason. When I look at the grid view, one task is in skipped state due to dag timeout. But that task did not get executed during the entire period until timeout.</p>
<p dir="auto">This happens randomly to any dag, to any task, at any time.</p>
<p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/3456652/184148510-fad37728-1a6a-4d07-9c8c-55038df6372a.png"><img src="https://user-images.githubusercontent.com/3456652/184148510-fad37728-1a6a-4d07-9c8c-55038df6372a.png" alt="Screenshot 2022-07-27 at 12 04 00 PM" style="max-width: 100%;"></a><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/3456652/184148550-2e7bf4fe-28a1-4199-8929-14a4531d5e10.png"><img width="1792" alt="Screenshot 2022-07-27 at 12 04 06 PM" src="https://user-images.githubusercontent.com/3456652/184148550-2e7bf4fe-28a1-4199-8929-14a4531d5e10.png" style="max-width: 100%;"></a><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/3456652/184148557-3cd26862-593c-417c-bcc4-0805c44d9cf1.png"><img width="1792" alt="Screenshot 2022-07-27 at 12 04 10 PM" src="https://user-images.githubusercontent.com/3456652/184148557-3cd26862-593c-417c-bcc4-0805c44d9cf1.png" style="max-width: 100%;"></a><br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/3456652/184148562-7b81dfcf-ac18-4517-8f70-165969429ea5.png"><img width="1792" alt="Screenshot 2022-07-27 at 12 04 13 PM" src="https://user-images.githubusercontent.com/3456652/184148562-7b81dfcf-ac18-4517-8f70-165969429ea5.png" style="max-width: 100%;"></a></p>
<h3 dir="auto">What you think should happen instead</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">How to reproduce</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">Ubuntu 20.04.3 LTS</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Virtualenv installation</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Anything else</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.2.2</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">We are using celery executor with Redis as broker.<br>
We are using default settings for celery.<br>
We are trying to test below case.</p>
<ul dir="auto">
<li>What happens when Redis or Worker Pod goes down?</li>
</ul>
<p dir="auto">Observations:</p>
<ul dir="auto">
<li>
<p dir="auto">We tried killing Redis Pod when one task was in <code class="notranslate">queued</code> state.</p>
</li>
<li>
<p dir="auto">We observed that task which was in <code class="notranslate">queued</code> state stays in <code class="notranslate">queued</code> state even after Redis Pod comes up.</p>
</li>
<li>
<p dir="auto">From Airflow UI we tried clearing task and run again. But still it was getting stuck at <code class="notranslate">queued</code> state only.</p>
</li>
<li>
<p dir="auto">Task message was received at <code class="notranslate">celery worker</code> . But worker is not starting executing the task.</p>
</li>
<li>
<p dir="auto">Let us know if we can try changing any celery or airflow config to avoid this issue.</p>
</li>
<li>
<p dir="auto">Also what is best practice to handle such cases.</p>
</li>
<li>
<p dir="auto">Please help here to avoid this case. As this is very critical if this happens in production.</p>
</li>
</ul>
<h3 dir="auto">What you think should happen instead</h3>
<p dir="auto">Task must not get stuck at <code class="notranslate">queued</code> state and it should start executing.</p>
<h3 dir="auto">How to reproduce</h3>
<p dir="auto">While task is in queued state. Kill the Redis pod.</p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">k8s</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Official Helm chart</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Anything else</h3>
<p dir="auto"><em>No response</em></p>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>