<p dir="auto">Airflow 1.10.9<br>
Celery 4.3.0</p>
<p dir="auto">Setting Celery 'worker_concurrency' is overriden by the autoscaling no matter what you configure.</p>
<p dir="auto">Problematic code is here:</p>
<p dir="auto"></p><div class="Box Box--condensed my-2">
  <div class="Box-header f6">
    <p class="mb-0 text-bold">
      <a href="https://github.com/apache/airflow/blob/6db66eaf927a50f2dcd85d0ae6033f7ea3699099/airflow/cli/commands/celery_command.py#L121">airflow/airflow/cli/commands/celery_command.py</a>
    </p>
    <p class="mb-0 color-fg-muted">
         Line 121
      in
      <a data-pjax="true" class="commit-tease-sha" href="/apache/airflow/commit/6db66eaf927a50f2dcd85d0ae6033f7ea3699099">6db66ea</a>
    </p>
  </div>
  <div itemprop="text" class="Box-body p-0 blob-wrapper blob-wrapper-embedded data">
    <table class="highlight tab-size mb-0 js-file-line-container" data-tab-size="8" data-paste-markdown-skip="">

        <tbody><tr class="border-0">
          <td id="L121" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="121"></td>
          <td id="LC121" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-s">'autoscale'</span>: <span class="pl-s1">autoscale</span>, </td>
        </tr>
    </tbody></table>
  </div>
</div>
<p></p>
<p dir="auto">(this is head but it's the same bug in the 1.10.x version of this file)</p>
<p dir="auto">You always get the default autoscaling setting with head if you don't set autoscaling at all but do set worker concurrency (which is 12 workers).</p>
<p dir="auto">Not adding "autoscale" to the dictionary when autoscaling is not set is enough to solve this (at least, I tested the worker_concurrency half of things, and the asked for concurrency is back with that change).</p>
<p dir="auto"><strong>Apache Airflow version</strong>: 1.10.9</p>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>OS</strong>: Debian GNU/Linux 10</li>
<li><strong>Kernel</strong>: Linux 4.19.0-8-amd64 <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="69689814" data-permission-text="Title is private" data-url="https://github.com/apache/airflow/issues/1" data-hovercard-type="pull_request" data-hovercard-url="/apache/airflow/pull/1/hovercard" href="https://github.com/apache/airflow/pull/1">#1</a> SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux</li>
</ul>
<p dir="auto"><strong>What happened</strong>:<br>
The <code class="notranslate">worker_autoscale</code> configuration option in <code class="notranslate">airflow.cfg</code> was overridded by the default value in <code class="notranslate">airflow/config_templates/default_airflow.cfg</code>  when left commented in the <code class="notranslate">airflow.cfg</code> file.</p>
<p dir="auto"><strong>What you expected to happen</strong>:<br>
The <code class="notranslate">worker_autoscale</code> option not being overridden by the default value.</p>
<p dir="auto"><strong>How to reproduce it</strong>:</p>
<ul dir="auto">
<li>Comment the <code class="notranslate">worker_autoscale</code> option in the <code class="notranslate">[celery]</code> section in <code class="notranslate">airflow.cfg</code>.</li>
<li>Spawn a worker by running <code class="notranslate">airflow worker</code></li>
<li>The celery worker will be spawned with the default autoscale option (16,12)</li>
</ul>
<p dir="auto"><strong>Anything else we need to know</strong>:</p>
<ul dir="auto">
<li>As the <code class="notranslate">worker_autoscale</code> option has priority over the <code class="notranslate">concurrency</code> option, the last one gets ignored even if autoscale is commented in the config file.</li>
<li>The <code class="notranslate">worker_autoscale</code> option is commented in  <code class="notranslate">airflow/config_templates/default_airflow.cfg</code> in the master branch (which fixes the issue) but not in the 1.10.9 release.</li>
</ul>