MongoDB 3 is slower than MongoDB 2
Mongodb 3 is slower than Mongodb 2 ?
Andrea Bozzettounread,May 15, 2015, 4:21:34 AM5/15/15to mongod...@googlegroups.comHi to all,  we have converted our application to MongoDB 3. We have seen some operations like insert are slower.We have made a test application. This webapp exposes a REST service called from a jmeter project.The jmeter runs 1500 thread in 15 secondsThe results  are indicated in the next table.  mongodb ver/ jdriver | Average | MAX---------------------------------------------2.6.9 / 2.13.1       |   68    | 9773.0.0 / 2.13.1       |  137    | 16792.6.9 / 3.0.0        |   78    | 12023.0.0 / 3.0.0        |  119    | 1500You can see, mongodb 2 is faster than mongodb 3.I attach the maven project, with  jmeter testIt's something wrong with my test ?Thank's a lotbyeAsya Kamskyunread,May 16, 2015, 7:25:59 PM5/16/15to mongodb-userYou are using jmeter?   How exactly?   At one point jmeter would send everything wrapped in db.eval which was way wrong, and of course it would be limited by that.Instead of attaching the project which I don't know if I can even read, why don't you describe exactly what you are testing?   > The jmeter runs 1500 thread in 15 secondsWhat do these threads do?In addition, I would suggesting testing against 3.0.3 rather than 3.0.0.Asya
Andrea Bozzettounread,May 18, 2015, 4:33:47 AM5/18/15to mongod...@googlegroups.comThe attachment is a maven project. The project create a webapp. This webapp exposes a REST service in POST. The REST service execute a query and save the json received in the POST in a MongoDB collection. I use jmeter only for send the json to REST service. To make some stress test jmeter runs 1500 thread in 15 seconds I did a new test using mongodb 3.03 and wiredtiger.mongodb ver/ jdriver        | Average | MAX
      ---------------------------------------------
      2.6.9 MMAPv1 / 2.13.1       |   68    | 977
      3.0.0 MMAPv1 / 2.13.1       |  137    | 1679
      2.6.9 MMAPv1 / 3.0.0        |   78    | 1202
      3.0.0 MMAPv1 / 3.0.0        |  119    | 1500
      3.0.3 wtiger / 2.13.1       |  184    | 1202
      3.0.3 wtiger / 3.0.0        |  165    | 874thanks a lotbyeAsya Kamskyunread,May 18, 2015, 6:06:08 AM5/18/15to mongodb-userAndrea:So the test appears to be multithreaded - how many cores on the server machine?  (1500 threads seems like a lot if you have, for example, only a few cores)...I'm a little surprised that mmap would be faster than wired tiger - however, regardless of threads, I would *not* expect 3.0 MMAP to ever be slower than 2.6 MMAP *if* the delay was in the database itself.I suspect the delay is in the way jmeter interacts with mongod - do you happen to have the logs from mongod during the time the test is running?   It would be illuminating to see what operations are running on the server and how long they are each taking.   If no operation takes less than 100ms it wouldn't be logged but you can increase the logging verbosity starting mongod with -v flag or by running this command on the shell:db.adminCommand({setParameter:1, logLevel:1})(after, don't forget to run it again with logLevel:0).Let me know what's in the logs after the test (probably don't need the full log but just showing some representative lines from it...)AsyaStephen Dillonunread,May 18, 2015, 6:37:56 AM5/18/15to mongod...@googlegroups.comIn my experience, 3.0 wiredTiger has been faster than 2.6.x when comparing inserts. I've not achieved the claimed 7+x performance boost in my use case but I vae seen better performance. Here are some other thoughts; beyond what the Mongo rep has covered. Is this a sharded cluster or a standalone instance? If it is a sharded cluster; did you upgrade from 2.6.x to 3.0 WT including a resync of the data into the new dbpath for WT or did you start testing with an empty data path? I ask because I wonder if you're 3.0 cluster is suffering from premature sharding. If you do not have any data in the new dbpath for 3.0 WT and you run insert tests, I can easily see how there could be overhead on the cluster once the balancer begins to move data. I've seen such performance degradation, across all versions, many times when the QA team wishes to perform tests against a multi shard cluster with no data. Andrea Bozzettounread,May 18, 2015, 7:36:28 AM5/18/15to mongod...@googlegroups.com"I suspect the delay is in the way jmeter interacts with mongod"Jmeter doesn't interacts with mongodb directly, JMeter call the REST Service. The REST Service interact with mongodb.Here the java code:@RequestMapping(method = RequestMethod.POST, value = "/newDocument", headers = "content-type=application/json")    public ResponseEntity<String> applicationUpsert(@RequestBody String json) {        HttpHeaders responseHeaders = new HttpHeaders();        responseHeaders.setContentType(MediaType.valueOf("application/json"));        try {            DB db = mongo.getDB("test");            Jongo jongo = new Jongo(db);            String query = "{'RegistryObjectList.ExtrinsicObject' : {$elemMatch : { 'ExternalIdentifier.value' : {$in: ['73387957557175793898']}, "                    + "'Slot.name': 'hash', 'ExternalIdentifier.identificationScheme': 'urn:uuid:2e82c1f6-a085-4c72-9da3-8640a32e42ab'}}}";            MongoCollection submission = jongo.getCollection("testPerf");            Iterable<String> documentsAsJson = submission.find(query).map(new JSONResultHandler());            Iterator<String> iter = documentsAsJson.iterator();            String s;            if (iter.hasNext()) {                while (iter.hasNext()) {                    s = iter.next();                }            }            DBObject jsonColl = (DBObject) JSON.parse(json);            WriteResult res = submission.insert(jsonColl);            return new ResponseEntity<String>("{\"response\": \"OK\"}",                    responseHeaders,                    HttpStatus.CREATED);        } catch (Exception e) {            log.error(e.getMessage(), e);            return new ResponseEntity<String>("{\"response\": \"KO\", \"errorMessage\": \"" + e.getMessage() + "\"}",                    responseHeaders,                    HttpStatus.INTERNAL_SERVER_ERROR);        }    }Jmeter divides the threads in 15 seconds, so the maximun threads per seconds is 200.A sample image of jmeter resultThanks a lotbyeAsya Kamskyunread,May 18, 2015, 5:29:25 PM5/18/15to mongodb-userUnfortunately, without mongod logs that I asked about I wouldn't be able to guess where the bottleneck is.AsyaAndrea Bozzettounread,May 19, 2015, 3:26:07 AM5/19/15to mongod...@googlegroups.comHi Asya,   this is the log of mongodb with log level 0 and 1ThanksByeAsya Kamskyunread,May 20, 2015, 12:29:42 AM5/20/15to mongodb-userGreat!  Now that you know how to get a logLevel:1 log of your test (this one was done with 3.0) you can do the same thing with 2.6 and compare them.Basically you are looking for groups of operations that are taking longer in 3.0 than 2.6.   If you see queries be slower, it could be because I see a couple of queries that are unindexed - are they taking less time in 2.6?   These sorts of details would usually make it possible to see what is responsible for the slower performance.Asyahenry oswaldunread,Jun 10, 2015, 10:52:18 AM6/10/15to mongod...@googlegroups.comWe are seeing something very similar after upgrading our replicaset to mongo3 and using the same mmap files we are seeing slower queries. Did you manage to find the root of the problem Andrea?Andrea Bozzettounread,Jun 11, 2015, 4:14:44 AM6/11/15to mongod...@googlegroups.comHi Henry,  by now we have stopped the test with MongoDB 3 and we are still using mongoDB 2.6.3.We don't known where is the problem with MongoDB 3.byeAsya Kamskyunread,Jun 17, 2015, 7:09:59 AM6/17/15to mongodb-userPlease describe your full setup, workload and observed performance issue in a new thread - that will make it easier to help you.
Asya

