<p>I'm working on a program that will be processing files that could potentially be 100GB or more in size. The files contain sets of variable length records. I've got a first implementation up and running and am now looking towards improving performance, particularly at doing I/O more efficiently since the input file gets scanned many times.</p>
<p>Is there a rule of thumb for using <code>mmap()</code> versus reading in blocks via C++'s <code>fstream</code> library? What I'd like to do is read large blocks from disk into a buffer, process complete records from the buffer, and then read more.</p>
<p>The <code>mmap()</code> code could potentially get very messy since <code>mmap</code>'d blocks need to lie on page sized boundaries (my understanding) and records could potentially lie across page boundaries. With <code>fstream</code>s, I can just seek to the start of a record and begin reading again, since we're not limited to reading blocks that lie on page sized boundaries.</p>
<p>How can I decide between these two options without actually writing up a complete implementation first? Any rules of thumb (e.g., <code>mmap()</code> is 2x faster) or simple tests?</p>

[{'answer_id': 46002, 'author': 'Leon Timmermans', 'author_id': 4727, 'author_profile': 'https://Stackoverflow.com/users/4727', 'pm_score': 3, 'selected': False, 'text': "<p>mmap should be faster, but I don't know how much. It very much depends on your code. If you use mmap it's best to mmap the whole file at once, that will make you life a lot easier. One potential problem is that if your file is bigger than 4GB (or in practice the limit is lower, often 2GB) you will need a 64bit architecture. So if you're using a 32 environment, you probably don't want to use it.</p>\n\n<p>Having said that, there may be a better route to improving performance. You said <i>the input file gets scanned many times</i>, if you can read it out in one pass and then be done with it, that could potentially be much faster.</p>\n"}, {'answer_id': 46005, 'author': 'Ben Collins', 'author_id': 3279, 'author_profile': 'https://Stackoverflow.com/users/3279', 'pm_score': 5, 'selected': False, 'text': '<p>mmap is <em>way</em> faster.  You might write a simple benchmark to prove it to yourself:</p>\n\n<pre><code>char data[0x1000];\nstd::ifstream in("file.bin");\n\nwhile (in)\n{\n  in.read(data, 0x1000);\n  // do something with data\n}\n</code></pre>\n\n<p>versus:</p>\n\n<pre><code>const int file_size=something;\nconst int page_size=0x1000;\nint off=0;\nvoid *data;\n\nint fd = open("filename.bin", O_RDONLY);\n\nwhile (off &lt; file_size)\n{\n  data = mmap(NULL, page_size, PROT_READ, 0, fd, off);\n  // do stuff with data\n  munmap(data, page_size);\n  off += page_size;\n}\n</code></pre>\n\n<p>Clearly, I\'m leaving out details (like how to determine when you reach the end of the file in the event that your file isn\'t a multiple of <code>page_size</code>, for instance), but it really shouldn\'t be much more complicated than this.</p>\n\n<p>If you can, you might try to break up your data into multiple files that can be mmap()-ed in whole instead of in part (much simpler).  </p>\n\n<p>A couple of months ago I had a half-baked implementation of a sliding-window mmap()-ed stream class for boost_iostreams, but nobody cared and I got busy with other stuff.  Most unfortunately, I deleted an archive of old unfinished projects a few weeks ago, and that was one of the victims :-(</p>\n\n<p><strong>Update</strong>: I should also add the caveat that this benchmark would look quite different in Windows because Microsoft implemented a nifty file cache that does most of what you would do with mmap in the first place.  I.e., for frequently-accessed files, you could just do std::ifstream.read() and it would be as fast as mmap, because the file cache would have already done a memory-mapping for you, and it\'s transparent.</p>\n\n<p><strong>Final Update</strong>: Look, people: across a lot of different platform combinations of OS and standard libraries and disks and memory hierarchies, I can\'t say for certain that the system call <code>mmap</code>, viewed as a black box, will always always always be substantially faster than <code>read</code>.  That wasn\'t exactly my intent, even if my words could be construed that way.  <strong>Ultimately, my point was that memory-mapped i/o is generally faster than byte-based i/o; this is still true</strong>.  If you find experimentally that there\'s no difference between the two, then the only explanation that seems reasonable to me is that your platform implements memory-mapping under the covers in a way that is advantageous to the performance of calls to <code>read</code>.  The only way to be absolutely certain that you\'re using memory-mapped i/o in a portable way is to use <code>mmap</code>.  If you don\'t care about portability and you can rely on the particular characteristics of your target platforms, then using <code>read</code> may be suitable without sacrificing measurably any performance.</p>\n\n<p><strong>Edit to clean up answer list:</strong>\n@jbl:</p>\n\n<blockquote>\n  <p>the sliding window mmap sounds\n  interesting. Can you say a little more\n  about it?</p>\n</blockquote>\n\n<p>Sure - I was writing a C++ library for Git (a libgit++, if you will), and I ran into a similar problem to this: I needed to be able to open large (very large) files and not have performance be a total dog (as it would be with <code>std::fstream</code>).</p>\n\n<p><code>Boost::Iostreams</code> already has a mapped_file Source, but the problem was that it was <code>mmap</code>ping whole files, which limits you to 2^(wordsize).  On 32-bit machines, 4GB isn\'t big enough.  It\'s not unreasonable to expect to have <code>.pack</code> files in Git that become much larger than that, so I needed to read the file in chunks without resorting to regular file i/o.  Under the covers of <code>Boost::Iostreams</code>, I implemented a Source, which is more or less another view of the interaction between <code>std::streambuf</code> and <code>std::istream</code>.  You could also try a similar approach by just inheriting <code>std::filebuf</code> into a <code>mapped_filebuf</code> and similarly, inheriting <code>std::fstream</code> into <code>a mapped_fstream</code>.  It\'s the interaction between the two that\'s difficult to get right.  <code>Boost::Iostreams</code> has some of the work done for you, and it also provides hooks for filters and chains, so I thought it would be more useful to implement it that way.</p>\n'}, {'answer_id': 46058, 'author': 'Douglas Leeder', 'author_id': 3978, 'author_profile': 'https://Stackoverflow.com/users/3978', 'pm_score': 2, 'selected': False, 'text': '<p>Perhaps you should pre-process the files, so each record is in a separate file (or at least that each file is a mmap-able size).</p>\n\n<p>Also could you do all of the processing steps for each record, before moving onto the next one? Maybe that would avoid some of the IO overhead?</p>\n'}, {'answer_id': 47464, 'author': 'Pat Notz', 'author_id': 825, 'author_profile': 'https://Stackoverflow.com/users/825', 'pm_score': 2, 'selected': False, 'text': "<p>This sounds like a good use-case for multi-threading... I'd think you could pretty easily setup one thread to be reading data while the other(s) process it.  That may be a way to dramatically increase the perceived performance.  Just a thought.</p>\n"}, {'answer_id': 67846, 'author': 'mlbrock', 'author_id': 9966, 'author_profile': 'https://Stackoverflow.com/users/9966', 'pm_score': 3, 'selected': False, 'text': "<p>I'm sorry Ben Collins lost his sliding windows mmap source code. That'd be nice to have in Boost.</p>\n\n<p>Yes, mapping the file is much faster. You're essentially using the the OS virtual memory subsystem to associate memory-to-disk and vice versa. Think about it this way: if the OS kernel developers could make it faster they would. Because doing so makes just about everything faster: databases, boot times, program load times, et cetera.</p>\n\n<p>The sliding window approach really isn't that difficult as multiple continguous pages can be mapped at once. So the size of the record doesn't matter so long as the largest of any single record will fit into memory. The important thing is managing the book-keeping.</p>\n\n<p>If a record doesn't begin on a getpagesize() boundary, your mapping has to begin on the previous page. The length of the region mapped extends from the first byte of the record (rounded down if necessary to the nearest multiple of getpagesize()) to the last byte of the record (rounded up to the nearest multiple of getpagesize()). When you're finished processing a record, you can unmap() it, and move on to the next.</p>\n\n<p>This all works just fine under Windows too using CreateFileMapping() and MapViewOfFile() (and GetSystemInfo() to get SYSTEM_INFO.dwAllocationGranularity --- not SYSTEM_INFO.dwPageSize).</p>\n"}, {'answer_id': 151819, 'author': 'Tim Cooper', 'author_id': 10592, 'author_profile': 'https://Stackoverflow.com/users/10592', 'pm_score': 6, 'selected': False, 'text': '<p>The main performance cost is going to be disk i/o.  "mmap()" is certainly quicker than istream, but the difference might not be noticeable because the disk i/o will dominate your run-times.</p>\n\n<p>I tried Ben Collins\'s code fragment (see above/below) to test his assertion that "mmap() is <em>way</em> faster" and found no measurable difference. See my comments on his answer.</p>\n\n<p>I would certainly <em>not</em> recommend separately mmap\'ing each record in turn unless your "records" are huge - that would be horribly slow, requiring 2 system calls for each record and possibly losing the page out of the disk-memory cache.....</p>\n\n<p>In your case I think mmap(), istream and the low-level open()/read() calls will all be about the same.   I would recommend mmap() in these cases:</p>\n\n<ol>\n<li>There is random access (not sequential) within the file, AND</li>\n<li>the whole thing fits comfortably in memory OR there is locality-of-reference within the file so that certain pages can be mapped in and other pages mapped out.  That way the operating system uses the available RAM to maximum benefit.</li>\n<li>OR if multiple processes are reading/working on the same file, then mmap() is fantastic because the processes all share the same physical pages.</li>\n</ol>\n\n<p>(btw - I love mmap()/MapViewOfFile()).</p>\n'}, {'answer_id': 151865, 'author': 'oz10', 'author_id': 14069, 'author_profile': 'https://Stackoverflow.com/users/14069', 'pm_score': 2, 'selected': False, 'text': '<p>I agree that mmap\'d file I/O is going to be faster, but while your benchmarking the code, shouldn\'t the counter example be <em>somewhat</em> optimized? </p>\n\n<p>Ben Collins wrote:</p>\n\n<pre><code>char data[0x1000];\nstd::ifstream in("file.bin");\n\nwhile (in)\n{\n    in.read(data, 0x1000);\n    // do something with data \n}\n</code></pre>\n\n<p>I would suggest also trying:</p>\n\n<pre><code>char data[0x1000];\nstd::ifstream iifle( "file.bin");\nstd::istream  in( ifile.rdbuf() );\n\nwhile( in )\n{\n    in.read( data, 0x1000);\n    // do something with data\n}\n</code></pre>\n\n<p>And beyond that, you might also try making the buffer size the same size as one page of virtual memory, in case 0x1000 is not the size of one page of virtual memory on your machine... IMHO mmap\'d file I/O still wins, but this should make things closer.</p>\n'}, {'answer_id': 695054, 'author': 'mike', 'author_id': 84322, 'author_profile': 'https://Stackoverflow.com/users/84322', 'pm_score': 2, 'selected': False, 'text': '<p>To my mind, using mmap() "just" unburdens the developer from having to write their own caching code. In a simple "read through file eactly once" case, this isn\'t going to be hard (although as mlbrock points out you still save the memory copy into process space), but if you\'re going back and forth in the file or skipping bits and so forth, I believe the kernel developers have <em>probably</em> done a better job implementing caching than I can...</p>\n'}, {'answer_id': 1897055, 'author': 'ony', 'author_id': 230744, 'author_profile': 'https://Stackoverflow.com/users/230744', 'pm_score': 1, 'selected': False, 'text': "<p>I think the greatest thing about mmap is potential for asynchronous reading with:</p>\n\n<pre><code>    addr1 = NULL;\n    while( size_left > 0 ) {\n        r = min(MMAP_SIZE, size_left);\n        addr2 = mmap(NULL, r,\n            PROT_READ, MAP_FLAGS,\n            0, pos);\n        if (addr1 != NULL)\n        {\n            /* process mmap from prev cycle */\n            feed_data(ctx, addr1, MMAP_SIZE);\n            munmap(addr1, MMAP_SIZE);\n        }\n        addr1 = addr2;\n        size_left -= r;\n        pos += r;\n    }\n    feed_data(ctx, addr1, r);\n    munmap(addr1, r);</code></pre>\n\n<p>Problem is that I can't find the right MAP_FLAGS to give a hint that this memory should be synced from file asap.\nI hope that MAP_POPULATE gives the right hint for mmap (i.e. it will not try to load all contents before return from call, but will do that in async. with feed_data). At least it gives better results with this flag even that manual states that it does nothing without MAP_PRIVATE since 2.6.23.</p>\n"}, {'answer_id': 6383253, 'author': 'Dietrich Epp', 'author_id': 82294, 'author_profile': 'https://Stackoverflow.com/users/82294', 'pm_score': 9, 'selected': True, 'text': '<p>I was trying to find the final word on mmap / read performance on Linux and I came across a nice post (<a href="http://marc.info/?l=linux-kernel&amp;m=95496636207616&amp;w=2" rel="noreferrer">link</a>) on the Linux kernel mailing list.  It\'s from 2000, so there have been many improvements to IO and virtual memory in the kernel since then, but it nicely explains the reason why <code>mmap</code> or <code>read</code> might be faster or slower.</p>\n\n<ul>\n<li>A call to <code>mmap</code> has more overhead than <code>read</code> (just like <code>epoll</code> has more overhead than <code>poll</code>, which has more overhead than <code>read</code>).  Changing virtual memory mappings is a quite expensive operation on some processors for the same reasons that switching between different processes is expensive.</li>\n<li>The IO system can already use the disk cache, so if you read a file, you\'ll hit the cache or miss it no matter what method you use.</li>\n</ul>\n\n<p>However,</p>\n\n<ul>\n<li>Memory maps are generally faster for random access, especially if your access patterns are sparse and unpredictable.</li>\n<li>Memory maps allow you to <em>keep</em> using pages from the cache until you are done.  This means that if you use a file heavily for a long period of time, then close it and reopen it, the pages will still be cached.  With <code>read</code>, your file may have been flushed from the cache ages ago.  This does not apply if you use a file and immediately discard it.  (If you try to <code>mlock</code> pages just to keep them in cache, you are trying to outsmart the disk cache and this kind of foolery rarely helps system performance).</li>\n<li>Reading a file directly is very simple and fast.</li>\n</ul>\n\n<p>The discussion of mmap/read reminds me of two other performance discussions:</p>\n\n<ul>\n<li><p>Some Java programmers were shocked to discover that nonblocking I/O is often slower than blocking I/O, which made perfect sense if you know that nonblocking I/O requires making more syscalls.</p></li>\n<li><p>Some other network programmers were shocked to learn that <code>epoll</code> is often slower than <code>poll</code>, which makes perfect sense if you know that managing <code>epoll</code> requires making more syscalls.</p></li>\n</ul>\n\n<p><strong>Conclusion:</strong> Use memory maps if you access data randomly, keep it around for a long time, or if you know you can share it with other processes (<code>MAP_SHARED</code> isn\'t very interesting if there is no actual sharing).  Read files normally if you access data sequentially or discard it after reading.  And if either method makes your program less complex, do <em>that</em>.  For many real world cases there\'s no sure way to show one is faster without testing your actual application and NOT a benchmark.</p>\n\n<p>(Sorry for necro\'ing this question, but I was looking for an answer and this question kept coming up at the top of Google results.)</p>\n'}, {'answer_id': 22697165, 'author': 'Community', 'author_id': -1, 'author_profile': 'https://Stackoverflow.com/users/-1', 'pm_score': 2, 'selected': False, 'text': '<p>I remember mapping a huge file containing a tree structure into memory years ago. I was amazed by the speed compared to normal de-serialization which involves lot of work in memory, like allocating tree nodes and setting pointers.\nSo in fact I was comparing a single call to mmap (or its counterpart on Windows)\nagainst many (MANY) calls to operator new and constructor calls. \nFor such kind of task, mmap is unbeatable compared to de-serialization.\nOf course one should look into boosts relocatable pointer for this.</p>\n'}, {'answer_id': 41419353, 'author': 'BeeOnRope', 'author_id': 149138, 'author_profile': 'https://Stackoverflow.com/users/149138', 'pm_score': 6, 'selected': False, 'text': '<p>There are lots of good answers here already that cover many of the salient points, so I\'ll just add a couple of issues I didn\'t see addressed directly above. That is, this answer shouldn\'t be considered a comprehensive of the pros and cons, but rather an addendum to other answers here.</p>\n<h2>mmap seems like magic</h2>\n<p>Taking the case where the file is already fully cached<sup>1</sup> as the baseline<sup>2</sup>, <code>mmap</code> might seem pretty much like <em>magic</em>:</p>\n<ol>\n<li><code>mmap</code> only requires 1 system call to (potentially) map the entire file, after which no more system calls are needed.</li>\n<li><code>mmap</code> doesn\'t require a copy of the file data from kernel to user-space.</li>\n<li><code>mmap</code> allows you to access the file &quot;as memory&quot;, including processing it with whatever advanced tricks you can do against memory, such as compiler auto-vectorization, <a href="https://en.wikipedia.org/wiki/SIMD" rel="noreferrer">SIMD</a> intrinsics, prefetching, optimized in-memory parsing routines, OpenMP, etc.</li>\n</ol>\n<p>In the case that the file is already in the cache, it seems impossible to beat: you just directly access the kernel page cache as memory and it can\'t get faster than that.</p>\n<p>Well, it can.</p>\n<h2>mmap is not actually magic because...</h2>\n<h3>mmap still does per-page work</h3>\n<p>A primary hidden cost of <code>mmap</code> vs <a href="http://man7.org/linux/man-pages/man2/read.2.html" rel="noreferrer"><code>read(2)</code></a> (which is really the comparable OS-level syscall for <em>reading blocks</em>) is that with <code>mmap</code> you\'ll need to do &quot;some work&quot; for every 4K page accessed in a new mapping, even though it might be hidden by the page-fault mechanism.</p>\n<p>For a example a typical implementation that just <code>mmap</code>s the entire file will need to fault-in so 100 GB / 4K = 25 million faults to read a 100 GB file. Now, these will be <a href="https://en.wikipedia.org/wiki/Page_fault#Minor" rel="noreferrer"><em>minor faults</em></a>, but 25 million page faults is still not going to be super fast. The cost of a minor fault is probably in the 100s of nanos in the best case.</p>\n<h3>mmap relies heavily on TLB performance</h3>\n<p>Now, you can pass <code>MAP_POPULATE</code> to <code>mmap</code> to tell it to set up all the page tables before returning, so there should be no page faults while accessing it. Now, this has the little problem that it also reads the entire file into RAM, which is going to blow up if you try to map a 100GB file - but let\'s ignore that for now<sup>3</sup>. The kernel needs to do <em>per-page work</em> to set up these page tables (shows up as kernel time). This ends up being a major cost in the <code>mmap</code> approach, and it\'s proportional to the file size (i.e., it doesn\'t get relatively less important as the file size grows)<sup>4</sup>.</p>\n<p>Finally, even in user-space accessing such a mapping isn\'t exactly free (compared to large memory buffers not originating from a file-based <code>mmap</code>) - even once the page tables are set up, each access to a new page is going to, conceptually, incur a TLB miss. Since <code>mmap</code>ing a file means using the page cache and its 4K pages, you again incur this cost 25 million times for a 100GB file.</p>\n<p>Now, the actual cost of these TLB misses depends heavily on at least the following aspects of your hardware: (a) how many 4K TLB enties you have and how the rest of the translation caching works performs (b) how well hardware prefetch deals with with the TLB - e.g., can prefetch trigger a page walk? (c) how fast and how parallel the page walking hardware is. On modern high-end x86 Intel processors, the page walking hardware is in general very strong: there are at least 2 parallel page walkers, a page walk can occur concurrently with continued execution, and hardware prefetching can trigger a page walk. So the TLB impact on a <em>streaming</em> read load is fairly low - and such a load will often perform similarly regardless of the page size. Other hardware is usually much worse, however!</p>\n<h2>read() avoids these pitfalls</h2>\n<p>The <a href="https://linux.die.net/man/2/read" rel="noreferrer"><code>read()</code></a> syscall, which is what generally underlies the &quot;block read&quot; type calls offered e.g., in C, C++ and other languages has one primary disadvantage that everyone is well-aware of:</p>\n<ul>\n<li>Every <code>read()</code> call of N bytes must copy N bytes from kernel to user space.</li>\n</ul>\n<p>On the other hand, it avoids most the costs above - you don\'t need to map in 25 million 4K pages into user space. You can usually <code>malloc</code> a single buffer small buffer in user space, and re-use that repeatedly for all your <code>read</code> calls. On the kernel side, there is almost no issue with 4K pages or TLB misses because all of RAM is usually linearly mapped using a few very large pages (e.g., 1 GB pages on x86), so the underlying pages in the page cache are covered very efficiently in kernel space.</p>\n<p>So basically you have the following comparison to determine which is faster for a single read of a large file:</p>\n<p><strong>Is the extra per-page work implied by the <code>mmap</code> approach more costly than the per-byte work of copying file contents from kernel to user space implied by using <code>read()</code>?</strong></p>\n<p>On many systems, they are actually approximately balanced. Note that each one scales with completely different attributes of the hardware and OS stack.</p>\n<p>In particular, the <code>mmap</code> approach becomes relatively faster when:</p>\n<ul>\n<li>The OS has fast minor-fault handling and especially minor-fault bulking optimizations such as fault-around.</li>\n<li>The OS has a good <code>MAP_POPULATE</code> implementation which can efficiently process large maps in cases where, for example, the underlying pages are contiguous in physical memory.</li>\n<li>The hardware has strong page translation performance, such as large TLBs, fast second level TLBs, fast and parallel page-walkers, good prefetch interaction with translation and so on.</li>\n</ul>\n<p>... while the <code>read()</code> approach becomes relatively faster when:</p>\n<ul>\n<li>The <code>read()</code> syscall has good copy performance. E.g., good <code>copy_to_user</code> performance on the kernel side.</li>\n<li>The kernel has an efficient (relative to userland) way to map memory, e.g., using only a few large pages with hardware support.</li>\n<li>The kernel has fast syscalls and a way to keep kernel TLB entries around across syscalls.</li>\n</ul>\n<p>The hardware factors above vary <em>wildly</em> across different platforms, even within the same family (e.g., within x86 generations and especially market segments) and definitely across architectures (e.g., ARM vs x86 vs PPC).</p>\n<p>The OS factors keep changing as well, with various improvements on both sides causing a large jump in the relative speed for one approach or the other. A recent list includes:</p>\n<ul>\n<li>Addition of fault-around, described above, which really helps the <code>mmap</code> case without <code>MAP_POPULATE</code>.</li>\n<li>Addition of fast-path <code>copy_to_user</code> methods in <code>arch/x86/lib/copy_user_64.S</code>, e.g., using <code>REP MOVQ</code> when it is fast, which really help the <code>read()</code> case.</li>\n</ul>\n<h2>Update after Spectre and Meltdown</h2>\n<p>The mitigations for the Spectre and Meltdown vulnerabilities considerably increased the cost of a system call. On the systems I\'ve measured, the cost of a &quot;do nothing&quot; system call (which is an estimate of the pure overhead of the system call, apart from any actual work done by the call) went from about 100 ns on a typical modern Linux system to about 700 ns. Furthermore, depending on your system, the <a href="https://en.wikipedia.org/wiki/Kernel_page-table_isolation" rel="noreferrer">page-table isolation</a> fix specifically for Meltdown can have additional downstream effects apart from the direct system call cost due to the need to reload TLB entries.</p>\n<p>All of this is a relative disadvantage for <code>read()</code> based methods as compared to <code>mmap</code> based methods, since <code>read()</code> methods must make one system call for each &quot;buffer size&quot; worth of data. You can\'t arbitrarily increase the buffer size to amortize this cost since using large buffers usually performs worse since you exceed the L1 size and hence are constantly suffering cache misses.</p>\n<p>On the other hand, with <code>mmap</code>, you can map in a large region of memory with <code>MAP_POPULATE</code> and the access it efficiently, at the cost of only a single system call.</p>\n<hr />\n<p><sup>1</sup> This more-or-less also includes the case where the file wasn\'t fully cached to start with, but where the OS read-ahead is good enough to make it appear so (i.e., the page is usually cached by the time you want it). This is a subtle issue though because the way read-ahead works is often quite different between <code>mmap</code> and <code>read</code> calls, and can be further adjusted by &quot;advise&quot; calls as described in <sup>2</sup>.</p>\n<p><sup>2</sup> ... because if the file is <em>not</em> cached, your behavior is going to be completely dominated by IO concerns, including how sympathetic your access pattern is to the underlying hardware - and all your effort should be in ensuring such access is as sympathetic as possible, e.g. via use of <a href="http://man7.org/linux/man-pages/man2/madvise.2.html" rel="noreferrer"><code>madvise</code></a> or <a href="https://linux.die.net/man/2/fadvise" rel="noreferrer"><code>fadvise</code></a> calls (and whatever application level changes you can make to improve access patterns).</p>\n<p><sup>3</sup> You could get around that, for example, by sequentially <code>mmap</code>ing in windows of a smaller size, say 100 MB.</p>\n<p><sup>4</sup> In fact, it turns out the <code>MAP_POPULATE</code> approach is (at least one some hardware/OS combination) only slightly faster than not using it, probably because the kernel is using <a href="https://lwn.net/Articles/588802/" rel="noreferrer">faultaround</a> - so the actual number of minor faults is reduced by a factor of 16 or so.</p>\n'}]
['https://Stackoverflow.com/questions/45972', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/2353001/']