Can't execute queries until end of atomic block in my data migration on django 1.7
Question: I have a pretty long data migration that I'm doing to correct an earlier bad
migration where some rows were created incorrectly. I'm trying to assign
values to a new column based upon old ones, however, sometimes this leads to
integrity errors. When this happens, I want to throw away the one that's
causing the integrity error

Here is a code snippet:

    
    
    def load_data(apps, schema_editor):
        MyClass = apps.get_model('my_app', 'MyClass')
    
        new_col_mapping = {old_val1: new_val1, ....}
    
        for inst in MyClass.objects.filter(old_col=c):
    
            try:
                inst.new_col = new_col_mapping[c]
                inst.save()
    
            except IntegrityError:
                inst.delete()
    

Then in the operations of my `Migration` class I do

    
    
    operations = [
        migrations.RunPython(load_data)
    ]
    

I get the following error when running the migration

    
    
    django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block
    

I get the feeling that doing

    
    
    with transaction.atomic():
    

somewhere is my solution but I'm not exactly sure where the right place is.
More importantly I'd like to understand WHY this is necessary


Answer: This is similar to the [example in the
docs](https://docs.djangoproject.com/en/1.8/topics/db/transactions/#controlling-
transactions-explicitly).

First, add the required import if you don't have it already.

    
    
    from django.db import transaction
    

Then wrap the code that might raise an integrity error in an atomic block.

    
    
    try:
        with transaction.atomic():
            inst.new_col = new_col_mapping[c]
            inst.save()
    except IntegrityError:
        inst.delete()
    

The reason for the error is explained in the warning block 'Avoid catching
exceptions inside atomic!' in the docs. Once Django encounters a database
error, it will roll back the atomic block. Attempting any more database
queries will cause a `TransactionManagementError`, which you are seeing. By
wrapping the code in an atomic block, only that code will be rolled back, and
you can execute queries outside of the block.

