<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.4.0</p>
<h3 dir="auto">What happened</h3>
<h2 dir="auto">ISSUE</h2>
<p dir="auto">I have run other dag files previously they all give this message even if they pass or fail.</p>
<h2 dir="auto">Goal</h2>
<p dir="auto">Get this error fixed</p>
<h2 dir="auto">Log that contains the error</h2>
<p dir="auto">I got this after running  simple dag</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="241adsgf1108
*** Log file does not exist: /opt/airflow/logs/dag_id=numpy_pandas/run_id=manual__2022-09-27T16:31:22.968544+00:00/task_id=print_the_context/attempt=1.log
*** Fetching from: http://241adsgf1108:8793/dag_id=numpy_pandas/run_id=manual__2022-09-27T16:31:22.968544+00:00/task_id=print_the_context/attempt=1.log
*** !!!! Please make sure that all your Airflow components (e.g. schedulers, webservers and workers) have the same 'secret_key' configured in 'webserver' section and time is synchronized on all your machines (for example with ntpd) !!!!!
****** See more at https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#secret-key
****** Failed to fetch log file from worker. Client error '403 FORBIDDEN' for url 'http://241adsgf1108:8793/dag_id=numpy_pandas/run_id=manual__2022-09-27T16:31:22.968544+00:00/task_id=print_the_context/attempt=1.log'
For more information check: https://httpstatuses.com/403"><pre class="notranslate"><code class="notranslate">241adsgf1108
*** Log file does not exist: /opt/airflow/logs/dag_id=numpy_pandas/run_id=manual__2022-09-27T16:31:22.968544+00:00/task_id=print_the_context/attempt=1.log
*** Fetching from: http://241adsgf1108:8793/dag_id=numpy_pandas/run_id=manual__2022-09-27T16:31:22.968544+00:00/task_id=print_the_context/attempt=1.log
*** !!!! Please make sure that all your Airflow components (e.g. schedulers, webservers and workers) have the same 'secret_key' configured in 'webserver' section and time is synchronized on all your machines (for example with ntpd) !!!!!
****** See more at https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#secret-key
****** Failed to fetch log file from worker. Client error '403 FORBIDDEN' for url 'http://241adsgf1108:8793/dag_id=numpy_pandas/run_id=manual__2022-09-27T16:31:22.968544+00:00/task_id=print_the_context/attempt=1.log'
For more information check: https://httpstatuses.com/403
</code></pre></div>
<h2 dir="auto">Commands</h2>
<ul dir="auto">
<li><code class="notranslate">docker build -t my-image-apache/airflow:latest-python3.8 . </code></li>
<li><code class="notranslate">docker-compose up </code></li>
</ul>
<h2 dir="auto">Environment</h2>
<ul dir="auto">
<li>AWS EC2</li>
<li>Ubuntu 20.04</li>
</ul>
<h3 dir="auto">What you think should happen instead</h3>
<h2 dir="auto">Folder Structure</h2>
<ul dir="auto">
<li>airflow /
<ul dir="auto">
<li>docker-compose.yml</li>
<li>Dockerfile</li>
<li>dags [FOLDER]/
<ul dir="auto">
<li>all_python_dag_files.py</li>
</ul>
</li>
</ul>
</li>
</ul>
<h3 dir="auto">How to reproduce</h3>
<h2 dir="auto">Files</h2>
<p dir="auto">my dag file</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import pendulum

from airflow import DAG
#from airflow.decorators import task
from airflow.operators.python import PythonOperator

with DAG(
    dag_id=&quot;numpy_pandas&quot;,
    schedule=None,
    start_date=pendulum.datetime(2021, 1, 1, tz=&quot;UTC&quot;),
    catchup=False,
    tags=[&quot;example&quot;],) as dag:

    def numpy_something():
        &quot;&quot;&quot;Print Numpy array.&quot;&quot;&quot;
        import numpy as np  # &lt;- THIS IS HOW NUMPY SHOULD BE IMPORTED IN THIS CASE
        import pandas as pd

        d = {'col1': [1, 2], 'col2': [3, 4]}
        df = pd.DataFrame(data=d)
        print(df)
        a = np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
        print(a)
        #a= 10
        return a

    run_this = PythonOperator(
        task_id=&quot;print_the_context&quot;,
        python_callable=numpy_something,
    )
"><pre class="notranslate"><code class="notranslate">import pendulum

from airflow import DAG
#from airflow.decorators import task
from airflow.operators.python import PythonOperator

with DAG(
    dag_id="numpy_pandas",
    schedule=None,
    start_date=pendulum.datetime(2021, 1, 1, tz="UTC"),
    catchup=False,
    tags=["example"],) as dag:

    def numpy_something():
        """Print Numpy array."""
        import numpy as np  # &lt;- THIS IS HOW NUMPY SHOULD BE IMPORTED IN THIS CASE
        import pandas as pd

        d = {'col1': [1, 2], 'col2': [3, 4]}
        df = pd.DataFrame(data=d)
        print(df)
        a = np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
        print(a)
        #a= 10
        return a

    run_this = PythonOperator(
        task_id="print_the_context",
        python_callable=numpy_something,
    )

</code></pre></div>
<h3 dir="auto">Operating System</h3>
<p dir="auto">Ubuntu 20.04.4 LTS</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<p dir="auto">dockerfile</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="FROM apache/airflow:latest-python3.8
COPY requirements.txt .
COPY personal_python_file.py /usr/local/airflow/dags/personal_python_file.py
RUN pip install -r requirements.txt"><pre class="notranslate"><code class="notranslate">FROM apache/airflow:latest-python3.8
COPY requirements.txt .
COPY personal_python_file.py /usr/local/airflow/dags/personal_python_file.py
RUN pip install -r requirements.txt
</code></pre></div>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Docker-Compose</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto">docker-compose.yml</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="---
version: '3'
x-airflow-common:
  &amp;airflow-common
  # In order to add custom dependencies or upgrade provider packages you can use your extended image.
  # Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
  # and uncomment the &quot;build&quot; line below, Then run `docker-compose build` to build the images.
  image: ${AIRFLOW_IMAGE_NAME:-my-image-apache/airflow:latest-python3.8} 
  ### SAME AS IN MY DOCKERFILE &quot;FROM apache/airflow:latest-python3.8&quot; BUT I CAN MODIFY IT AS THE IAMGE THAT I GNERATE IS THE IMPORTANT ONE
  # my-image-apache/airflow:latest-python3.8}
  # build: .
  environment:
    &amp;airflow-common-env
    AIRFLOW__CORE__EXECUTOR: CeleryExecutor
    AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
    # For backward compatibility, with Airflow &lt;2.3
    AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
    AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql://airflow:airflow@postgres/airflow
    AIRFLOW__CELERY__BROKER_URL: redis://:@redis:6379/0
    AIRFLOW__CORE__FERNET_KEY: ''
    AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
    AIRFLOW__CORE__LOAD_EXAMPLES: 'false'
    AIRFLOW__API__AUTH_BACKENDS: 'airflow.api.auth.backend.basic_auth'
    _PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}

    # I have added these
    AIRFLOW__CORE__ENABLE_XCOM_PICKLING: 'true'
    #AIRFLOW__CORE__DAGS_FOLDER: /opt/airflow/dags
    #connect local and docker conatiner DAGS folers
    #AIRFLOW__CORE__DAGS_FOLDER: ./dags
  

    # AIRFLOW__CORE__PLUGINS_FOLDER: /opt/airflow/plugins
    # AIRFLOW__CORE__LOGGING_CONFIG_CLASS: airflow.utils.log.logging_config.DEFAULT_LOGGING_CONFIG
    # AIRFLOW__CORE__LOGGING_LEVEL: INFO

  volumes:
    - ./dags/:/opt/airflow/dags
    - ./logs/:/opt/airflow/logs
    - ./plugins/:/opt/airflow/plugins
  user: &quot;${AIRFLOW_UID:-50000}:0&quot;
  depends_on:
    &amp;airflow-common-depends-on
    redis:
      condition: service_healthy
    postgres:
      condition: service_healthy

services:
  postgres:
    image: postgres:13
    environment:
      POSTGRES_USER: airflow
      POSTGRES_PASSWORD: airflow
      POSTGRES_DB: airflow
    volumes:
      - postgres-db-volume:/var/lib/postgresql/data
    healthcheck:
      test: [&quot;CMD&quot;, &quot;pg_isready&quot;, &quot;-U&quot;, &quot;airflow&quot;]
      interval: 5s
      retries: 5
    restart: always

  redis:
    image: redis:latest
    expose:
      - 6379
    healthcheck:
      test: [&quot;CMD&quot;, &quot;redis-cli&quot;, &quot;ping&quot;]
      interval: 5s
      timeout: 30s
      retries: 50
    restart: always

  airflow-webserver:
    &lt;&lt;: *airflow-common
    command: webserver
    ports:
      - 8080:8080
    healthcheck:
      test: [&quot;CMD&quot;, &quot;curl&quot;, &quot;--fail&quot;, &quot;http://localhost:8080/health&quot;]
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    stdin_open: true
    tty: true
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully
    volumes:
      - ./airflow:/usr/local/airflow

  airflow-scheduler:
    &lt;&lt;: *airflow-common
    command: scheduler
    healthcheck:
      test: [&quot;CMD-SHELL&quot;, 'airflow jobs check --job-type SchedulerJob --hostname &quot;$${HOSTNAME}&quot;']
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-worker:
    &lt;&lt;: *airflow-common
    command: celery worker
    healthcheck:
      test:
        - &quot;CMD-SHELL&quot;
        - 'celery --app airflow.executors.celery_executor.app inspect ping -d &quot;celery@$${HOSTNAME}&quot;'
      interval: 10s
      timeout: 10s
      retries: 5
    environment:
      &lt;&lt;: *airflow-common-env
      # Required to handle warm shutdown of the celery workers properly
      # See https://airflow.apache.org/docs/docker-stack/entrypoint.html#signal-propagation
      DUMB_INIT_SETSID: &quot;0&quot;
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-triggerer:
    &lt;&lt;: *airflow-common
    command: triggerer
    healthcheck:
      test: [&quot;CMD-SHELL&quot;, 'airflow jobs check --job-type TriggererJob --hostname &quot;$${HOSTNAME}&quot;']
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-init:
    &lt;&lt;: *airflow-common
    entrypoint: /bin/bash
    # yamllint disable rule:line-length
    command:
      - -c
      - |
        function ver() {
          printf &quot;%04d%04d%04d%04d&quot; $${1//./ }
        }
        airflow_version=$$(AIRFLOW__LOGGING__LOGGING_LEVEL=INFO &amp;&amp; gosu airflow airflow version)
        airflow_version_comparable=$$(ver $${airflow_version})
        min_airflow_version=2.2.0
        min_airflow_version_comparable=$$(ver $${min_airflow_version})
        if (( airflow_version_comparable &lt; min_airflow_version_comparable )); then
          echo
          echo -e &quot;\033[1;31mERROR!!!: Too old Airflow version $${airflow_version}!\e[0m&quot;
          echo &quot;The minimum Airflow version supported: $${min_airflow_version}. Only use this or higher!&quot;
          echo
          exit 1
        fi
        if [[ -z &quot;${AIRFLOW_UID}&quot; ]]; then
          echo
          echo -e &quot;\033[1;33mWARNING!!!: AIRFLOW_UID not set!\e[0m&quot;
          echo &quot;If you are on Linux, you SHOULD follow the instructions below to set &quot;
          echo &quot;AIRFLOW_UID environment variable, otherwise files will be owned by root.&quot;
          echo &quot;For other operating systems you can get rid of the warning with manually created .env file:&quot;
          echo &quot;    See: https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#setting-the-right-airflow-user&quot;
          echo
        fi
        one_meg=1048576
        mem_available=$$(($$(getconf _PHYS_PAGES) * $$(getconf PAGE_SIZE) / one_meg))
        cpus_available=$$(grep -cE 'cpu[0-9]+' /proc/stat)
        disk_available=$$(df / | tail -1 | awk '{print $$4}')
        warning_resources=&quot;false&quot;
        if (( mem_available &lt; 4000 )) ; then
          echo
          echo -e &quot;\033[1;33mWARNING!!!: Not enough memory available for Docker.\e[0m&quot;
          echo &quot;At least 4GB of memory required. You have $$(numfmt --to iec $$((mem_available * one_meg)))&quot;
          echo
          warning_resources=&quot;true&quot;
        fi
        if (( cpus_available &lt; 2 )); then
          echo
          echo -e &quot;\033[1;33mWARNING!!!: Not enough CPUS available for Docker.\e[0m&quot;
          echo &quot;At least 2 CPUs recommended. You have $${cpus_available}&quot;
          echo
          warning_resources=&quot;true&quot;
        fi
        if (( disk_available &lt; one_meg * 10 )); then
          echo
          echo -e &quot;\033[1;33mWARNING!!!: Not enough Disk space available for Docker.\e[0m&quot;
          echo &quot;At least 10 GBs recommended. You have $$(numfmt --to iec $$((disk_available * 1024 )))&quot;
          echo
          warning_resources=&quot;true&quot;
        fi
        if [[ $${warning_resources} == &quot;true&quot; ]]; then
          echo
          echo -e &quot;\033[1;33mWARNING!!!: You have not enough resources to run Airflow (see above)!\e[0m&quot;
          echo &quot;Please follow the instructions to increase amount of resources available:&quot;
          echo &quot;   https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#before-you-begin&quot;
          echo
        fi
        mkdir -p /sources/logs /sources/dags /sources/plugins
        chown -R &quot;${AIRFLOW_UID}:0&quot; /sources/{logs,dags,plugins}
        exec /entrypoint airflow version
    # yamllint enable rule:line-length
    environment:
      &lt;&lt;: *airflow-common-env
      _AIRFLOW_DB_UPGRADE: 'true'
      _AIRFLOW_WWW_USER_CREATE: 'true'
      _AIRFLOW_WWW_USER_USERNAME: ${_AIRFLOW_WWW_USER_USERNAME:-airflow}
      _AIRFLOW_WWW_USER_PASSWORD: ${_AIRFLOW_WWW_USER_PASSWORD:-airflow}
      _PIP_ADDITIONAL_REQUIREMENTS: ''
    user: &quot;0:0&quot;
    volumes:
      - .:/sources

  airflow-cli:
    &lt;&lt;: *airflow-common
    profiles:
      - debug
    environment:
      &lt;&lt;: *airflow-common-env
      CONNECTION_CHECK_MAX_COUNT: &quot;0&quot;
    # Workaround for entrypoint issue. See: https://github.com/apache/airflow/issues/16252
    command:
      - bash
      - -c
      - airflow

  # You can enable flower by adding &quot;--profile flower&quot; option e.g. docker-compose --profile flower up
  # or by explicitly targeted on the command line e.g. docker-compose up flower.
  # See: https://docs.docker.com/compose/profiles/
  flower:
    &lt;&lt;: *airflow-common
    command: celery flower
    profiles:
      - flower
    ports:
      - 5555:5555
    healthcheck:
      test: [&quot;CMD&quot;, &quot;curl&quot;, &quot;--fail&quot;, &quot;http://localhost:5555/&quot;]
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

volumes:
  postgres-db-volume:
"><pre class="notranslate"><code class="notranslate">---
version: '3'
x-airflow-common:
  &amp;airflow-common
  # In order to add custom dependencies or upgrade provider packages you can use your extended image.
  # Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
  # and uncomment the "build" line below, Then run `docker-compose build` to build the images.
  image: ${AIRFLOW_IMAGE_NAME:-my-image-apache/airflow:latest-python3.8} 
  ### SAME AS IN MY DOCKERFILE "FROM apache/airflow:latest-python3.8" BUT I CAN MODIFY IT AS THE IAMGE THAT I GNERATE IS THE IMPORTANT ONE
  # my-image-apache/airflow:latest-python3.8}
  # build: .
  environment:
    &amp;airflow-common-env
    AIRFLOW__CORE__EXECUTOR: CeleryExecutor
    AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
    # For backward compatibility, with Airflow &lt;2.3
    AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow
    AIRFLOW__CELERY__RESULT_BACKEND: db+postgresql://airflow:airflow@postgres/airflow
    AIRFLOW__CELERY__BROKER_URL: redis://:@redis:6379/0
    AIRFLOW__CORE__FERNET_KEY: ''
    AIRFLOW__CORE__DAGS_ARE_PAUSED_AT_CREATION: 'true'
    AIRFLOW__CORE__LOAD_EXAMPLES: 'false'
    AIRFLOW__API__AUTH_BACKENDS: 'airflow.api.auth.backend.basic_auth'
    _PIP_ADDITIONAL_REQUIREMENTS: ${_PIP_ADDITIONAL_REQUIREMENTS:-}

    # I have added these
    AIRFLOW__CORE__ENABLE_XCOM_PICKLING: 'true'
    #AIRFLOW__CORE__DAGS_FOLDER: /opt/airflow/dags
    #connect local and docker conatiner DAGS folers
    #AIRFLOW__CORE__DAGS_FOLDER: ./dags
  

    # AIRFLOW__CORE__PLUGINS_FOLDER: /opt/airflow/plugins
    # AIRFLOW__CORE__LOGGING_CONFIG_CLASS: airflow.utils.log.logging_config.DEFAULT_LOGGING_CONFIG
    # AIRFLOW__CORE__LOGGING_LEVEL: INFO

  volumes:
    - ./dags/:/opt/airflow/dags
    - ./logs/:/opt/airflow/logs
    - ./plugins/:/opt/airflow/plugins
  user: "${AIRFLOW_UID:-50000}:0"
  depends_on:
    &amp;airflow-common-depends-on
    redis:
      condition: service_healthy
    postgres:
      condition: service_healthy

services:
  postgres:
    image: postgres:13
    environment:
      POSTGRES_USER: airflow
      POSTGRES_PASSWORD: airflow
      POSTGRES_DB: airflow
    volumes:
      - postgres-db-volume:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "airflow"]
      interval: 5s
      retries: 5
    restart: always

  redis:
    image: redis:latest
    expose:
      - 6379
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 30s
      retries: 50
    restart: always

  airflow-webserver:
    &lt;&lt;: *airflow-common
    command: webserver
    ports:
      - 8080:8080
    healthcheck:
      test: ["CMD", "curl", "--fail", "http://localhost:8080/health"]
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    stdin_open: true
    tty: true
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully
    volumes:
      - ./airflow:/usr/local/airflow

  airflow-scheduler:
    &lt;&lt;: *airflow-common
    command: scheduler
    healthcheck:
      test: ["CMD-SHELL", 'airflow jobs check --job-type SchedulerJob --hostname "$${HOSTNAME}"']
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-worker:
    &lt;&lt;: *airflow-common
    command: celery worker
    healthcheck:
      test:
        - "CMD-SHELL"
        - 'celery --app airflow.executors.celery_executor.app inspect ping -d "celery@$${HOSTNAME}"'
      interval: 10s
      timeout: 10s
      retries: 5
    environment:
      &lt;&lt;: *airflow-common-env
      # Required to handle warm shutdown of the celery workers properly
      # See https://airflow.apache.org/docs/docker-stack/entrypoint.html#signal-propagation
      DUMB_INIT_SETSID: "0"
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-triggerer:
    &lt;&lt;: *airflow-common
    command: triggerer
    healthcheck:
      test: ["CMD-SHELL", 'airflow jobs check --job-type TriggererJob --hostname "$${HOSTNAME}"']
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

  airflow-init:
    &lt;&lt;: *airflow-common
    entrypoint: /bin/bash
    # yamllint disable rule:line-length
    command:
      - -c
      - |
        function ver() {
          printf "%04d%04d%04d%04d" $${1//./ }
        }
        airflow_version=$$(AIRFLOW__LOGGING__LOGGING_LEVEL=INFO &amp;&amp; gosu airflow airflow version)
        airflow_version_comparable=$$(ver $${airflow_version})
        min_airflow_version=2.2.0
        min_airflow_version_comparable=$$(ver $${min_airflow_version})
        if (( airflow_version_comparable &lt; min_airflow_version_comparable )); then
          echo
          echo -e "\033[1;31mERROR!!!: Too old Airflow version $${airflow_version}!\e[0m"
          echo "The minimum Airflow version supported: $${min_airflow_version}. Only use this or higher!"
          echo
          exit 1
        fi
        if [[ -z "${AIRFLOW_UID}" ]]; then
          echo
          echo -e "\033[1;33mWARNING!!!: AIRFLOW_UID not set!\e[0m"
          echo "If you are on Linux, you SHOULD follow the instructions below to set "
          echo "AIRFLOW_UID environment variable, otherwise files will be owned by root."
          echo "For other operating systems you can get rid of the warning with manually created .env file:"
          echo "    See: https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#setting-the-right-airflow-user"
          echo
        fi
        one_meg=1048576
        mem_available=$$(($$(getconf _PHYS_PAGES) * $$(getconf PAGE_SIZE) / one_meg))
        cpus_available=$$(grep -cE 'cpu[0-9]+' /proc/stat)
        disk_available=$$(df / | tail -1 | awk '{print $$4}')
        warning_resources="false"
        if (( mem_available &lt; 4000 )) ; then
          echo
          echo -e "\033[1;33mWARNING!!!: Not enough memory available for Docker.\e[0m"
          echo "At least 4GB of memory required. You have $$(numfmt --to iec $$((mem_available * one_meg)))"
          echo
          warning_resources="true"
        fi
        if (( cpus_available &lt; 2 )); then
          echo
          echo -e "\033[1;33mWARNING!!!: Not enough CPUS available for Docker.\e[0m"
          echo "At least 2 CPUs recommended. You have $${cpus_available}"
          echo
          warning_resources="true"
        fi
        if (( disk_available &lt; one_meg * 10 )); then
          echo
          echo -e "\033[1;33mWARNING!!!: Not enough Disk space available for Docker.\e[0m"
          echo "At least 10 GBs recommended. You have $$(numfmt --to iec $$((disk_available * 1024 )))"
          echo
          warning_resources="true"
        fi
        if [[ $${warning_resources} == "true" ]]; then
          echo
          echo -e "\033[1;33mWARNING!!!: You have not enough resources to run Airflow (see above)!\e[0m"
          echo "Please follow the instructions to increase amount of resources available:"
          echo "   https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html#before-you-begin"
          echo
        fi
        mkdir -p /sources/logs /sources/dags /sources/plugins
        chown -R "${AIRFLOW_UID}:0" /sources/{logs,dags,plugins}
        exec /entrypoint airflow version
    # yamllint enable rule:line-length
    environment:
      &lt;&lt;: *airflow-common-env
      _AIRFLOW_DB_UPGRADE: 'true'
      _AIRFLOW_WWW_USER_CREATE: 'true'
      _AIRFLOW_WWW_USER_USERNAME: ${_AIRFLOW_WWW_USER_USERNAME:-airflow}
      _AIRFLOW_WWW_USER_PASSWORD: ${_AIRFLOW_WWW_USER_PASSWORD:-airflow}
      _PIP_ADDITIONAL_REQUIREMENTS: ''
    user: "0:0"
    volumes:
      - .:/sources

  airflow-cli:
    &lt;&lt;: *airflow-common
    profiles:
      - debug
    environment:
      &lt;&lt;: *airflow-common-env
      CONNECTION_CHECK_MAX_COUNT: "0"
    # Workaround for entrypoint issue. See: https://github.com/apache/airflow/issues/16252
    command:
      - bash
      - -c
      - airflow

  # You can enable flower by adding "--profile flower" option e.g. docker-compose --profile flower up
  # or by explicitly targeted on the command line e.g. docker-compose up flower.
  # See: https://docs.docker.com/compose/profiles/
  flower:
    &lt;&lt;: *airflow-common
    command: celery flower
    profiles:
      - flower
    ports:
      - 5555:5555
    healthcheck:
      test: ["CMD", "curl", "--fail", "http://localhost:5555/"]
      interval: 10s
      timeout: 10s
      retries: 5
    restart: always
    depends_on:
      &lt;&lt;: *airflow-common-depends-on
      airflow-init:
        condition: service_completed_successfully

volumes:
  postgres-db-volume:

</code></pre></div>
<h3 dir="auto">Anything else</h3>
<p dir="auto">Tried Already:</p>
<ul dir="auto">
<li><a href="https://stackoverflow.com/questions/59591008/airflow-giving-log-file-does-not-exist-error-while-running-on-docker" rel="nofollow">https://stackoverflow.com/questions/59591008/airflow-giving-log-file-does-not-exist-error-while-running-on-docker</a></li>
<li><a href="https://forum.astronomer.io/t/log-file-does-not-exist/277" rel="nofollow">https://forum.astronomer.io/t/log-file-does-not-exist/277</a> (can not access jira link)</li>
</ul>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
<h3 dir="auto">Discussed in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="4404000" data-permission-text="Title is private" data-url="https://github.com/apache/airflow/discussions/26490" data-hovercard-type="discussion" data-hovercard-url="/apache/airflow/discussions/26490/hovercard" href="https://github.com/apache/airflow/discussions/26490">#26490</a></h3>
<div type="discussions-op-text" dir="auto">
<p dir="auto"><sup>Originally posted by <strong>emredjan</strong> September 19, 2022</sup></p>
<h3 dir="auto">Apache Airflow version</h3>
<p dir="auto">2.4.0</p>
<h3 dir="auto">What happened</h3>
<p dir="auto">When running tasks on a remote celery worker, webserver fails to fetch logs from the machine, giving a '403 - Forbidden' error on version 2.4.0. This behavior does not happen on 2.3.3, where the remote logs are retrieved and displayed successfully.<br>
The <code class="notranslate">webserver / secret_key</code> configuration is the same in all nodes (the config files are synced), and their time is synchronized using a central NTP server, making the solution in the warning message not applicable.</p>
<p dir="auto">My limited analysis pointed to the <code class="notranslate">serve_logs.py</code> file, and the flask request object that's passed to it, but couldn't find the root cause.</p>
<h3 dir="auto">What you think should happen instead</h3>
<p dir="auto">It should fetch and show remote celery worker logs on the webserver UI correctly, as it did in previous versions.</p>
<h3 dir="auto">How to reproduce</h3>
<p dir="auto">Use airflow version 2.4.0<br>
Use CeleryExecutor with RabbitMQ<br>
Use a separate Celery worker machine<br>
Run a dag/task on the remote worker<br>
Try to display task log on the web UI</p>
<h3 dir="auto">Operating System</h3>
<p dir="auto">Red Hat Enterprise Linux 8.6 (Ootpa)</p>
<h3 dir="auto">Versions of Apache Airflow Providers</h3>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="apache-airflow-providers-celery==3.0.0
apache-airflow-providers-common-sql==1.1.0
apache-airflow-providers-ftp==3.0.0
apache-airflow-providers-hashicorp==3.0.0
apache-airflow-providers-http==3.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-microsoft-mssql==3.0.0
apache-airflow-providers-mysql==3.0.0
apache-airflow-providers-odbc==3.0.0
apache-airflow-providers-sftp==3.0.0
apache-airflow-providers-sqlite==3.0.0
apache-airflow-providers-ssh==3.0.0"><pre class="notranslate"><code class="notranslate">apache-airflow-providers-celery==3.0.0
apache-airflow-providers-common-sql==1.1.0
apache-airflow-providers-ftp==3.0.0
apache-airflow-providers-hashicorp==3.0.0
apache-airflow-providers-http==3.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-microsoft-mssql==3.0.0
apache-airflow-providers-mysql==3.0.0
apache-airflow-providers-odbc==3.0.0
apache-airflow-providers-sftp==3.0.0
apache-airflow-providers-sqlite==3.0.0
apache-airflow-providers-ssh==3.0.0
</code></pre></div>
<h3 dir="auto">Deployment</h3>
<p dir="auto">Virtualenv installation</p>
<h3 dir="auto">Deployment details</h3>
<p dir="auto">Using CeleryExecutor / rabbitmq with 2 servers</p>
<h3 dir="auto">Anything else</h3>
<p dir="auto">All remote task executions has the same problem.</p>
<h3 dir="auto">Are you willing to submit PR?</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> Yes I am willing to submit a PR!</li>
</ul>
<h3 dir="auto">Code of Conduct</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> I agree to follow this project's <a href="https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md">Code of Conduct</a></li>
</ul>
</div>