title: How to squash/split SQLAlchemy migrations files for minimal service downtime?
body: <p>I have a SQLAlchemy/Flask application that is being accessed publicly by millions of people.</p>
  
  <p>Every 2 weeks, I release a new version of this application and I run database migrations using <code>python manage.py db upgrade</code>. This runs any new migration files that were created since the last migration file was run.</p>
  
  <p>The problem with this approach is that since my database tables are so large, simple migration operations like adding a column can take 15-20 minutes each. So stopping the application, running the migrations, updating the application code then restarting the application can cause a lengthy downtime. I'm trying to minimize this downtime.</p>
  
  <p>My strategy to minimize the downtime is to run as many migration files (or as many fractional portions of migration files) before the deploy, without stopping the service. Clearly there are some operations that can be done this way. </p>
  
  <p>For example <a href="http://alembic.zzzcomputing.com/en/latest/ops.html" rel="nofollow noreferrer"><strong>op.create_table()</strong></a> and <strong>op.add_column()</strong> can safely be run before updating the code without interfering with the operation of the application. <strong>op.delete_column</strong> cannot.</p>
  
  <p>Is there any automated way to separate out the migration commands that can be run without affecting the application's operation vs those that do affect the application's operation? Currently I have to manually go through these migrations by hand and perform brain surgery on them. It's painful. But I can't be the only one to ever experience this problem. So what's the solution that other people use?</p>