title: MS-Access Database getting very large during inserts
body: <p>I have a database which I regularly need to import large amounts of data into via some python scripts. Compacted, the data for a single months imports takes about 280mb, but during the import file size swells to over a gb.</p>
  
  <p>Given the 2gb size limit on mdb files, this is a bit of a concern. Apart from breaking the inserts into chunks and compacting inbetween each, are there any techniques for avoiding the increase in file size?</p>
  
  <p>Note that no temporary tables are being created/deleted during the process: just inserts into existing tables.</p>
  
  <p>And to forstall the inevitable comments: yes, I am required to store this data in Access 2003. No, I can't upgrade to Access 2007.</p>
  
  <p>If it could help, I could preprocess in sqlite.</p>
  
  <p>Edit:</p>
  
  <p>Just to add some further information (some already listed in my comments):</p>
  
  <ul>
  <li>The data is being generated in Python on a table by table basis, and then all of the records for that table batch inserted via odbc</li>
  <li>All processing is happening in Python: all the mdb file is doing is storing the data</li>
  <li>All of the fields being inserted are valid fields (none are being excluded due to unique key violations, etc.)</li>
  </ul>
  
  <p>Given the above, I'll be looking into how to disable row level locking via odbc and considering presorting the data and/or removing then reinstating indexes. Thanks for the suggestions.</p>
  
  <p>Any further suggestions still welcome.</p>