Python + MongoDB - Cursor iteration too slow
Question: I'm actually working in a search engine project.  
We are working with python + mongoDb.  
I'm having the following problem:

I have a pymongo cursor after excecuting a find() command to the mongo db.  
The pymongo cursor have around 20k results.

I have noticed that the iteration over the pymongo cursor is really slow
compared with a normal iteration over for example a list of the same size.

I did a little benchmark:

-iteration over a list of 20k strings: 0.001492 seconds  
-iteration over a pymongo cursor with 20k results: 1.445343 seconds

The difference is really a lot. Maybe not a problem with this amounts of
results, but if I have millons of results the time would be unacceptable.

Has anyone got an idea of why pymongo cursors are too slow to iterate?  
Any idea of how can I iterate the cursor in less time?

Some extra info:

  * Python v2.6
  * PyMongo v1.9
  * MongoDB v1.6 32 bits


Answer: Is your pymongo installation using the included [C
extensions](http://api.mongodb.org/python/current/installation.html#dependencies-
for-installing-c-extensions-on-unix)?

    
    
    >>> import pymongo
    >>> pymongo.has_c()
    True
    

I spent most of last week trying to debug a moderate-sized query and
corresponding processing that took 20 seconds to run. Once the C extensions
were installed, the whole same process took roughly a second.

To install the C extensions in Debian, install the python development headers
before running easy install. In my case, I also had to remove the old version
of pymongo. Note that this will compile a binary from C, so you need all the
usual tools. (GCC, etc)

    
    
    # on ubuntu with pip
    $ sudo pip uninstall pymongo
    $ sudo apt-get install python-dev build-essential
    $ sudo pip install pymongo
    

