title: Celery + Flask + Docker, consumer: Cannot connect to amqp://admin:**@rabbit:5672/myhost: failed to resolve broker hostname
body: <p><strong>Background</strong></p>
  <p>I am building a web application that uses Flask for the backend framework. The application uses Celery to handle all the time-consuming tasks as background tasks as to not block the backend thread. I use RabbitMQ as the message broker for Celery workers. I bundled each service using docker-compose.</p>
  <p><strong>Problem</strong></p>
  <p>The app has been working well until the past few days, and all of sudden, Celery workers keep failing to connect to the message broker with the error message <code>[ERROR/MainProcess] consumer: Cannot connect to amqp://admin:**@rabbit:5672/myhost: failed to resolve broker hostname.</code></p>
  <p><strong>Directory structure and code</strong></p>
  <p>I put together files and directories for minimally reproducible example.</p>
  <pre><code>debug/
  ├── code
  │   ├── dev.Dockerfile
  │   ├── my_app
  │   │   ├── celery_app.py
  │   │   ├── config.py
  │   │   ├── extensions.py
  │   │   ├── __init__.py
  │   │   ├── my_tasks.py
  │   │   └── test_app.py
  │   └── requirements.txt
  └── docker-compose_dev.yml
  
  </code></pre>
  <p>docker-compose_dev.yml</p>
  <pre><code>version: &quot;3.7&quot;
  services:
    rabbit:
      image: rabbitmq:3.8.5-management
      ports:
        - '15673:15672' # in case user has rabbitMQ installed on host
      expose:
        - &quot;5672&quot;
      environment:
        - RABBITMQ_DEFAULT_USER=admin
        - RABBITMQ_DEFAULT_PASS=mypass
        - RABBITMQ_DEFAULT_VHOST=myhost
  
    non_working_worker:
      build:
        context: ./code
        dockerfile: dev.Dockerfile
      command: &quot;celery worker -A my_app.celery_app:app -l info&quot;
      volumes:
        - ./code:/code
      links:
        - rabbit
  
    working_worker:
      build:
        context: ./code
        dockerfile: dev.Dockerfile
      command: &quot;celery worker -A my_app.my_tasks:app -l info&quot;
      volumes:
        - ./code:/code
      links:
        - rabbit
  </code></pre>
  <p>dev.Dockerfile</p>
  <pre><code>FROM continuumio/miniconda3
  
  # Make /backend working directory; flask code lives here
  WORKDIR /code
  
  # Install from requirements.txt using pip
  COPY requirements.txt requirements.txt
  RUN pip install -r requirements.txt
  RUN rm requirements.txt
  </code></pre>
  <p>requirements.txt</p>
  <pre><code>luigi==2.8.11
  plotnine==0.7.0
  celery==4.4.6
  flask==1.1.2
  flask-cors
  flask-socketio
  Flask-Mail
  eventlet
  </code></pre>
  <p>test_app.py</p>
  <pre><code>import eventlet
  eventlet.monkey_patch()
  
  from flask import Flask
  from my_app.extensions import celery
  
  def create_app():
      &quot;&quot;&quot;
      Application factory. Create application here.
      &quot;&quot;&quot;
      app = Flask(__name__)
      app.config.from_object(&quot;my_app.config&quot;)
  
      return app
  
  def init_celery(app=None):
      &quot;&quot;&quot;
      Initialize Celery App
      &quot;&quot;&quot;
      app = app or create_app()
      app.config.from_object(&quot;my_app.config&quot;)
  
      # Set celery worker configuration
      # Use this to load config information from flask config file
      celery.conf.broker_url = app.config[&quot;CELERY_BROKER_URL&quot;]
      celery.conf.result_backend = app.config[&quot;CELERY_RESULT_BACKEND&quot;]
  
      class ContextTask(celery.Task):
          &quot;&quot;&quot;Make celery tasks work with Flask app context&quot;&quot;&quot;
  
          def __call__(self, *args, **kwargs):
              with app.app_context():
                  return self.run(*args, **kwargs)
  
      celery.Task = ContextTask
  
      return celery
  </code></pre>
  <p>config.py</p>
  <pre><code># RabbitMQ
  CELERY_BROKER_URL='pyamqp://admin:mypass@rabbit/myhost'
  CELERY_RESULT_BACKEND='rpc://'
  </code></pre>
  <p>extensions.py</p>
  <pre><code>from celery import Celery
  
  celery = Celery()
  </code></pre>
  <p>celery_app.py</p>
  <pre><code>from my_app.test_app import init_celery
  
  app = init_celery()
  </code></pre>
  <p>my_tasks.py</p>
  <pre><code>from celery import Celery
  
  
  app = Celery()
  
  app.conf.broker_url = 'pyamqp://admin:mypass@rabbit/myhost'
  app.conf.result_backend = 'rpc://'
  </code></pre>
  <p><strong>What I've tried</strong></p>
  <p>Followings are the things I've tried, but didn't work.</p>
  <ol>
  <li>RabbitMQ isn't launching properly?
  <ul>
  <li>a. It launches properly with given username, password, and vhost. (can check using the management plugin @ localhost:15673)</li>
  </ul>
  </li>
  <li>RabbitMQ launches after the Celery workers start, so the workers can't find the broker?
  <ul>
  <li>a. Celery has <code>retry</code> feature, so it will keep on retrying until message broker is up running.</li>
  </ul>
  </li>
  <li>Network issue?
  <ul>
  <li>a. I've tried with/without <code>links</code> to specify service name alias, but still didn't work.</li>
  <li>b. Note I've already specified broker name as <code>rabbit</code> as specified in the <code>config.py</code> file instead of <code>localhost</code></li>
  <li>c. I've tried using both the default network <code>docker-compose</code> creates, and custom network, but both failed.</li>
  </ul>
  </li>
  <li>Interestingly, Celery app instance in <code>my_tasks.py</code> works (it's named as <code>working_worker</code> in the docker-compose file), but Celery app instance in the Flask factory pattern does not work (it'a named as <code>non_working_worker</code> in the compose file)
  <ul>
  <li>a. Again, it shows that RabbitMQ is working fine, but something funky is going on with the Flask factory pattern style Celery app instantiation.</li>
  </ul>
  </li>
  </ol>
  <p>I spent past few days trying to fix this issue and searching for similar problems on internet, but had no luck doing so.</p>
  <p>I know it's a fairly long post, but any help/suggestions would greatly be appreciated.</p>
  <p>docker-compose version</p>
  <pre><code>docker-compose version 1.25.3, build d4d1b42b
  docker-py version: 4.1.0
  CPython version: 3.7.5
  OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019
  </code></pre>
  <p>docker version</p>
  <pre><code>Client: Docker Engine - Community
   Version:           19.03.12
   API version:       1.40
   Go version:        go1.13.10
   Git commit:        48a66213fe
   Built:             Mon Jun 22 15:45:36 2020
   OS/Arch:           linux/amd64
   Experimental:      false
  
  Server: Docker Engine - Community
   Engine:
    Version:          19.03.12
    API version:      1.40 (minimum version 1.12)
    Go version:       go1.13.10
    Git commit:       48a66213fe
    Built:            Mon Jun 22 15:44:07 2020
    OS/Arch:          linux/amd64
    Experimental:     false
   containerd:
    Version:          1.2.13
    GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
   runc:
    Version:          1.0.0-rc10
    GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
   docker-init:
    Version:          0.18.0
    GitCommit:        fec3683
  
  </code></pre>