   Should I use memory-mapped files here?
  At work we process some massive proprietary-structured binary files. Within the files is a tree structure, with file offsets used as 'pointers.' To read a tree node, I read the entire node from the disk, overlay a struct on top of it, and go from there. To access any child nodes or data requires that another file seek and read be done before continuing. (Obviously, arranging the tree in a certain way can help alleviate this somewhat.)When I was getting my feet wet with it, I was lazy and just mmap()'d the entire file into memory. (The only quirk was I had to fix up file offsets by adding the base of the mmap()'d region to them.) However, not everyone has as much RAM as I do, and these files can get to be quite big. I do not want my program to fail if the user's system is memory taxed and it cannot reserve the necessary memory to map the entire file.My question is: is it possible to use memory-mapped files here while being mindful of one's memory footprint in this case? The worst case is that I'm jumping around quite a bit in a file. The only possible thing I can think of is constantly moving my view of the file around to be only what I'm working on - but internally that has to be seeking around in the file anyway, right? It seems that memory-mapped files seem to be far more useful for firehose-style I/O, where you dump a large amount of data to/from a file all at once. But I do not have much experience with them, and was looking for a second opinion.Thanks!   
  The behavior of memory mapped files depends on the underlying operating system; and seeing as you're using mmap() that narrows it down to only about a few hundred POSIX compatible OSs.   ;  Alright, I was referring to the family of memory-mapped functions colloquially as mmap(). My platform is Windows.   ;  I don't think allocating the entire file into RAM is a good idea if the files can be huge and the clients can lack the necesary amount of it.Is high access speed a requieremnt for this program? Try caching chunks of the file from one offset to another (the size for the cache could be set based on clients RAM availability). Or just access the nodes directly from the file in the hard-disk.   ;  Assuming a 32-bit NT based kernel, physical or even virtual memory probably isn't going to be a problem, since a memory mapped file is already backed by a file on disk, so the kernel won't bother allocated space in the page file. Your limiting factor is going to be the address space size. With default DLL basing that gives you about the range from 0x2000 0000 to 0x6fff fffff or about 1.25 GB as the maximum size of a file you can process. This is assuming that you don't use libraries that create their own heaps or do any fancy memory management on your own. However, with worse case assumptions I wouldn't want to rely on anything greater than 512 MB mapping properly.   