Upgrading Django from 1.8.6 to 1.9--django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet; importing models issue
Question: I'm trying to upgrade from Django 1.8.6 to 1.9 but I've been running into
trouble with trying to get the project to build and run properly. I've changed
numerous things necessary for the upgrade, like making an `apps.py` file in
`my_app` and defining Configs and including their dotted paths in
`INSTALLED_APPS` in `settings.py` (and changing `INSTALLED_APPS` to a list),
but the same error I get every time is:

    
    
    django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
    

For some background info, the project uses Docker Compose. The Dockerfile used
for the `web` container which runs the Django server itself utilizes `paver`
to start up so that when `docker-compose up` is run, the following commands
are executed:

    
    
    ./manage.py makemigrations --noinput
    ./manage.py migrate --noinput
    ./manage.py collectstatic --noinput
    pip install --upgrade pip
    pip install --upgrade -r requirements.txt
    ./manage.py runserver 0.0.0.0:8000
    

To my knowledge, none of the dependencies are incompatible with Django 1.9 so
I'm not sure if that's the issue here. The only dependency that I initially
thought could have been incompatible was `django_hstore`, but it was updated
for compatibility shortly after 1.9 was officially released. So unless the
creator of `django_hstore` is mistaken or lying (which I doubt), I can't
really think of any incompatible dependencies. The database backend used is
`django.db.backends.postgresql_psycopg2`. There is also a `worker` container
that uses the same Dockerfile as `web` and is used to run Celery. The full
error traceback from the Django `worker` container is below:

    
    
    worker_1 | Traceback (most recent call last):
    worker_1 |   File "/usr/local/bin/celery", line 11, in <module>
    worker_1 |     sys.exit(main())
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/__main__.py", line 30, in main
    worker_1 |     main()
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/bin/celery.py", line 81, in main
    worker_1 |     cmd.execute_from_commandline(argv)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/bin/celery.py", line 770, in execute_from_commandline
    worker_1 |     super(CeleryCommand, self).execute_from_commandline(argv)))
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/bin/base.py", line 309, in execute_from_commandline
    worker_1 |     argv = self.setup_app_from_commandline(argv)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/bin/base.py", line 469, in setup_app_from_commandline
    worker_1 |     self.app = self.find_app(app)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/bin/base.py", line 489, in find_app
    worker_1 |     return find_app(app, symbol_by_name=self.symbol_by_name)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/app/utils.py", line 238, in find_app
    worker_1 |     sym = imp(app)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/celery/utils/imports.py", line 101, in import_from_cwd
    worker_1 |     return imp(module, package=package)
    worker_1 |   File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    worker_1 |     __import__(name)
    worker_1 |   File "/code/my_app/tasks.py", line 3, in <module>
    worker_1 |     from taskman.celery import app, DBTask
    worker_1 |   File "/code/taskman/celery.py", line 6, in <module>
    worker_1 |     from utils.db.clearblackbox import rm_invalid_blackbox
    worker_1 |   File "/code/utils/db/clearblackbox.py", line 9, in <module>
    worker_1 |     django.setup()
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
    worker_1 |     apps.populate(settings.INSTALLED_APPS)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
    worker_1 |     app_config = AppConfig.create(entry)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 142, in create
    worker_1 |     app_module = import_module(app_name)
    worker_1 |   File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
    worker_1 |     __import__(name)
    worker_1 |   File "/code/utils/db/blackboxquery.py", line 2, in <module>
    worker_1 |     from my_app.models import BlackBox, DataPoint, Value, SourceInfo, FormatString, Argument
    worker_1 |   File "/code/my_app/models.py", line 11, in <module>
    worker_1 |     class Value(models.Model):
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 94, in __new__
    worker_1 |     app_config = apps.get_containing_app_config(module)
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 239, in get_containing_app_config
    worker_1 |     self.check_apps_ready()
    worker_1 |   File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 124, in check_apps_ready
    worker_1 |     raise AppRegistryNotReady("Apps aren't loaded yet.")
    worker_1 | django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
    

I've seen a number of similar questions that had the same AppRegistryNotReady
exception when upgrading to 1.9 but in my case I can specifically tell it's
because one of my apps whose AppConfig is in `apps.py` imports models when it
is recommended to not do so. The Django 1.9 documentation on application setup
says:

> At this stage, your code shouldn’t import any models!
>
> In other words, your applications’ root packages and the modules that define
> your application configuration classes shouldn’t import any models, even
> indirectly.
>
> Strictly speaking, Django allows importing models once their application
> configuration is loaded. However, in order to avoid needless constraints on
> the order of INSTALLED_APPS, it’s strongly recommended not import any models
> at this stage.

Unfortunately, the documentation seems to offer no alternative for importing
models during the setup stage, which is a shame since I can't actually avoid
importing models. Specifically, in my `celery.py`, I have a `Task` subclass
named `DBTask` whose `on_error` callback uses a module function to remove an
invalid database insertion. That module, `clearblackbox.py`, imports models
since it needs to call `delete()` on the invalid model that was inserted into
the database. The `DBTask` class is used as the base class for the main
database insertion task named `insertBlackboxIntoDatabaseTask`. Since I can't
get around importing models at the `setup()` stage, what else can I do to get
past this error and be able to run my server again?

**_EDIT:_** I was wondering if I had any unnecessary Configs so I got rid of
all but two: `my_app.apps.TasksConfig`, whose `name` field points to my
`tasks.py` containing the definition of my database insertion task, and
`taskman.celery.CeleryConfig`, which overrides `ready` such that it auto-
detects tasks from among `INSTALLED_APPS`. Though I put
`taskman.celery.CeleryConfig` in `INSTALLED_APPS`, the error I get now is

    
    
    ImportError: No module named CeleryConfig
    

This happens if I put `import django` and `django.setup()` in either
`celery.py` or `clearblackbox.py` in an attempt to solve the
`AppRegistryNotReady` exception that is happening due to importing models
during setup.


Answer: What ended up working for me was getting rid of all Configs in `apps.py`
except for the `TasksConfig`, then modifying the imports in `clearblackbox.py`
and `celery.py`. In `clearblackbox.py`, I moved the model import into the
function itself instead of leaving it at the top and in `celery.py`, I moved
the import of the function from `clearblackbox.py` into the `on_failure`
definition for `DBTask` rather than leave the import at the top. After doing
this, putting the dotted paths to the `TasksConfig` and `CeleryConfig` ended
up working.

