title: pymongo.errors.BulkWriteError: batch op errors occurred (MongoDB 3.4.2, pymongo 3.4.0, python 2.7.13)
body: <p>I am migrating several hundred million tweets of the format <code>{'id_str': , 'created_at': , 'text': }</code> from text files into MongoDB using pymongo. A collection is created for each user to store his/her tweets. The insertion method I am using is <code>insert_many()</code>. It often runs into <code>BulkWriteError</code>.</p>
  
  <pre><code>Traceback (most recent call last):
    File "pipeline.py", line 105, in &lt;module&gt;
      timeline_db, meta_db, negative_db, log_col, dir_path)
    File "/media/haitao/Storage/twitter_pipeline/migrate_old.py", line 134, in migrate_dir
      timeline_db[user_id].insert_many(utility.temporal_sort(statuses))
    File "/home/haitao/anaconda3/envs/py27/lib/python2.7/site-packages/pymongo/collection.py", line 711, in insert_many
      blk.execute(self.write_concern.document)
    File "/home/haitao/anaconda3/envs/py27/lib/python2.7/site-packages/pymongo/bulk.py", line 493, in execute
      return self.execute_command(sock_info, generator, write_concern)
    File "/home/haitao/anaconda3/envs/py27/lib/python2.7/site-packages/pymongo/bulk.py", line 331, in execute_command
      raise BulkWriteError(full_result)
  pymongo.errors.BulkWriteError: batch op errors occurred
  </code></pre>
  
  <p>This error seems to occur when there are duplicates of keys, which should not be the case for here. Are there other things that I can check to solve this issue?</p>
  
  <p>Thanks in advance!</p>