Python Flask: RQ Worker raising KeyError because of environment variable
Question: I'm trying to setup a redis queue and a worker to process the queue with my
flask app. I'm implementing this to handle a task that sends emails. I'm a
little confused because it appears that the stack trace is saying that my
'APP_SETTINGS' environment variable is not set when it is in fact set.

Prior to starting up the app, redis or the worker, I set APP_SETTINGS:

    
    
    export APP_SETTINGS="project.config.DevelopmentConfig"
    

However, when an item gets added to the queue, here's the stack trace:

    
    
    17:00:00 *** Listening on default...
    17:00:59 default: project.email.sendMailInBG(<flask_mail.Message object at 0x7fc930e1c3d0>) (aacf9546-5558-4db8-9232-5f36c25d521b)
    17:01:00 KeyError: 'APP_SETTINGS'
    Traceback (most recent call last):
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/worker.py", line 588, in perform_job
        rv = job.perform()
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/job.py", line 498, in perform
        self._result = self.func(*self.args, **self.kwargs)
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/job.py", line 206, in func
        return import_attribute(self.func_name)
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/utils.py", line 150, in import_attribute
        module = importlib.import_module(module_name)
      File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
      File "/home/tony/pyp-launch/project/__init__.py", line 24, in <module>
        app.config.from_object(os.environ['APP_SETTINGS'])
      File "/home/tony/pyp-launch/venv/lib/python2.7/UserDict.py", line 40, in __getitem__
        raise KeyError(key)
    KeyError: 'APP_SETTINGS'
    Traceback (most recent call last):
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/worker.py", line 588, in perform_job
        rv = job.perform()
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/job.py", line 498, in perform
        self._result = self.func(*self.args, **self.kwargs)
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/job.py", line 206, in func
        return import_attribute(self.func_name)
      File "/home/tony/pyp-launch/venv/local/lib/python2.7/site-packages/rq/utils.py", line 150, in import_attribute
        module = importlib.import_module(module_name)
      File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
        __import__(name)
      File "/home/tony/pyp-launch/project/__init__.py", line 24, in <module>
        app.config.from_object(os.environ['APP_SETTINGS'])
      File "/home/tony/pyp-launch/venv/lib/python2.7/UserDict.py", line 40, in __getitem__
        raise KeyError(key)
    KeyError: 'APP_SETTINGS'
    17:01:00 Moving job to u'failed' queue
    17:01:00 
    17:01:00 *** Listening on default...
    

**email.py**

    
    
    from flask.ext.mail import Message
    
    from project import app, mail
    from redis import Redis
    from rq import use_connection, Queue
    
    q = Queue(connection=Redis())
    
    
    def send_email(to, subject, template, emailable):
    
        if emailable==True:   
            msg = Message(
                subject,
                recipients=[to],
                html=template,
                sender=app.config['MAIL_DEFAULT_SENDER']
            )
            q.enqueue(sendMailInBG, msg)
    
        else:
            print("no email sent, emailable set to: " + str(emailable))
    
    def sendMailInBG(msgContent):
    
        with app.test_request_context():
            mail.send(msgContent)
    

**worker.py**

    
    
    import os
    
    import redis
    from rq import Worker, Queue, Connection
    
    listen = ['default']
    
    redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')
    
    conn = redis.from_url(redis_url)
    
    if __name__ == '__main__':
        with Connection(conn):
            worker = Worker(list(map(Queue, listen)))
            worker.work()
    

I'd really appreciate another set of eyes on this. I can't for the life of me
figure out what's going on here.


Answer: Thanks to the prompting of @danidee, I discovered that the environment
variables need to be defined in each terminal. Hence, APP_SETTINGS was defined
for the actual app, but not for the worker.

The solution was to set APP_SETTINGS in the worker terminal.

