MongoDB map-reduce impacts insert performance on source collection I have a "hot" MongoDB(4.0.5) capped collection with a timestamp (and several other) indexes.  Many applications (Java, NodeJS &amp; C++) insert (never update) records into this collection at a rate of ~50 inserts/second.  Normally these inserts are very quick and take only a few milliseconds.

When I run a map-reduce query on the most recent 5 minutes of data, calls to insert new records into the source collection are blocked or delayed.  New inserts during the map-reduce call can be delayed > 10 seconds which is roughly the time that it takes to run the map-reduce operation.  The output from the map-reduce operation is written to the same database.

What can I do to tune MongoDB or the map-reduce operation so that inserts can continue to flow into the capped collection without delay while the map-reduce operation runs?  I need to get the summary analysis in near real-time so I don't want to wait until the insert load is reduced before running the analysis.

I do not believe that this is a system level resource constraint.  I have monitored CPU, network, i/o and memory during the map-reduce call and CPU idle time remains > 50% and network and i/o latency remains low.  IO/wait times remain under 5%.
Maybe setting the nonAtomic: true flag could help.

Or you can try the Map-Reduce alternatives:


Views
Tailable Cursors
Change Streams