How can I upload 1000s of nodes per hour to a live drupal 7 site and avoid deadlocks? Not too long ago I wrote about deadlock here: PDOException: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock;

Despite everything my development team tries to do, we still get errors like this:


  PDOException: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found   when trying to get lock; try restarting transaction: INSERT INTO     {location_instance} (nid, vid, uid, genid, lid) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4); Array ( [:db_insert_placeholder_0] => 1059 [:db_insert_placeholder_1] => 1059 [:db_insert_placeholder_2] => 0 [:db_insert_placeholder_3] => cck:field_item_location:1059 [:db_insert_placeholder_4] => 1000 ) in location_save_locations() (line 974 of /var/www/website.com/sites/all/modules/location/location.module).


Despite the specific table in that example, we get this error on other tables. 

Here is my situation. I have taken a large University project. At any given time there are 50,000 campus residents who use the system daily.  In addition to that, I am migrating 100s of 1000s of items of content both manually and through custom module code (migration from the old university data) to this new Drupal 7 site.

This error is killing us, to the point where we are almost ready to scrap the last years worth of work and go with something else if Drupal cannot handle this type of load.

But that is more or less my question -  How can Drupal handle this type of load?  How can I organize my work flow to be able to handle this much activity? Is this a Drupal issue? A database issue?

Specifically, I am running Ubuntu, LAMP stack 16GB RAM. I am open to any suggestion whether it be Drupal related, database related, server config related, or a different work flow to work within Drupal's capabilities, so feel free to suggest anything if you have experience with this much activity.
I work for Stanford university and have been doing similar things. We constantly have to load over 100,000+ nodes on a regular basis. We have been working on our own custom loading code for 2 years now have been able to speed up the process quite a big using pcntl_fork. The only thing you have to remember is to close all socket connection before invoking the fork. For example you have to close you mysql connection, memcache connection and even mongo connection. Drupal will automatically create new connections when one doesn't exists. As far as the deadlock problem, we were able to fix that issue by putting innodb_locks_unsafe_for_binlog = 1.