MongoDB not that faster than MySQL?
Question: I discovered mongodb some months ago,and after reading this
[post](http://www.vedana.it/it/component/content/article/9-linux/62-testing-
mongodb-vs-mysql-with-python-scripting-under-linux), I thought mongodb was
really faster than mysql, so I decided to build my own bench, the problem is
that I do not have the same result than the above post's author, especially
for quering the database : mongodb seems to be slower than MyISAM tables.
Could you have a look to my python code, may be there is something wrong in it
:

    
    
    from datetime import datetime
    import random
    import MySQLdb
    import pymongo
    
    mysql_db=MySQLdb.connect(user="me",passwd="mypasswd",db="test_kv")
    c=mysql_db.cursor()
    
    connection = pymongo.Connection()
    mongo_db = connection.test
    kvtab = mongo_db.kvtab
    
    nb=1000000
    thelist=[]
    for i in xrange(nb):
        thelist.append((str(random.random()),str(random.random())))
    t1=datetime.now()
    
    for k,v in thelist:
        c.execute("INSERT INTO key_val_tab (k,v) VALUES ('" + k + "','" + v + "')")
    
    dt=datetime.now() - t1
    print 'MySQL insert elapse :',dt
    
    t1=datetime.now()
    for i in xrange(nb):
        c.execute("select * FROM key_val_tab WHERE k='" + random.choice(thelist)[0] + "'")
        result=c.fetchone()
    
    dt=datetime.now() - t1
    print 'MySQL select elapse :',dt
    
    
    t1=datetime.now()
    
    for k,v in thelist:
        kvtab.insert({"key":k,"value":v})
    
    dt=datetime.now() - t1
    print 'Mongodb insert elapse :',dt
    kvtab.ensure_index('key')
    t1=datetime.now()
    for i in xrange(nb):
        result=kvtab.find_one({"key":random.choice(thelist)[0]})
    
    dt=datetime.now() - t1
    print 'Mongodb select elapse :',dt
    

Notes:

  * both MySQL and mongodb are on locahost.
  * both MySQL and mongodb has the 'key' column indexed

MySQL Table:

    
    
    CREATE TABLE IF NOT EXISTS `key_val_tab` (
      `k` varchar(24) NOT NULL,
      `v` varchar(24) NOT NULL,
      KEY `kindex` (`k`)
    ) ENGINE=MyISAM DEFAULT CHARSET=latin1;
    

Versions are:

  * MySQL: 5.1.41
  * mongodb : 1.8.3
  * python : 2.6.5
  * pymongo : 2.0.1
  * Linux : Ubuntu 2.6.32 32Bits with PAE
  * Hardware : Desktop core i7 2.93 Ghz

Results (for 1 million inserts/selects) :

    
    
    MySQL insert elapse : 0:02:52.143803
    MySQL select elapse : 0:04:43.675914
    Mongodb insert elapse : 0:00:49.038416  -> mongodb much faster for insert
    Mongodb select elapse : 0:05:10.409025  -> ...but slower for quering (thought was the opposite)
    


Answer: Sigh. These kind of benchmarks, and I use the term loosely in this case,
usually break down from the very start. MySQL isn't a "slower" database than
MongoDB. One is a relational database, the other a NoSQL document store. They
will/should be faster in the functional areas that they were designed to
cover. In the case of MySQL (or any RDBMS) and MongoDB this overlap isn't as
big as a lot of people assume it is. It's the same kind of broken apples and
oranges comparison you get with Redis vs. MongoDB discussions.

There are so many variables (app functional requirements, hardware resources,
concurrency, configuration, scalability, etc.) to consider that any benchmark
or article that ends with "MongoDB is faster than MySQL" or vice versa is
generalizing results to the point of uselessness.

If you want to do benchmark first define a strict set of functional
requirements and business rules and then implement them as efficiently as
possible on both persistence solutions. The result will be that one is faster
than the other and in almost all cases the faster approach has some relevant
downsides that might still make the slower solution more viable depending on
requirements.

All this is ignoring that the benchmark above doesn't simulate any sort of
real world scenario. There wont be a lot of apps doing max throughput inserts
without any sort of threading/concurrency (which impacts performance on most
storage solutions significantly).

Finally, comparing inserts like this is a little broken too. MongoDB can
achieve amazing insert throughput with fire and forget bulk inserts or can be
orders of magnitude slower with fsynced, replicated writes. The thing here is
that MongoDB offers you a choice where MySQL doesn't (or less so). So here the
comparison only make sense of the business requirements allow for fire and
forget type writes (Which boil down to, "I hope it works, but no biggy if it
didn't")

TL;DR stop doing simple throughput benchmarks. They're almost always useless.

