title: Avoid memory fragmentation when memory pools are a bad idea
body: <p>I am developing a C++ application, where the program run endlessly, allocating and freeing millions of strings (char*) over time. And RAM usage is a serious consideration in the program. This results in RAM usage getting higher and higher over time. I think the problem is heap fragmentation. And I really need to find a solution.</p>
  <p><a href="https://i.stack.imgur.com/nLyPJ.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/nLyPJ.png" alt="Ram usage Report 1" /></a></p>
  <p>You can see in the image, after millions of allocation and freeing in the program, the usage is just increasing. And the way I am testing it, I know for a fact that the data it stores is not increasing. I can guess that you will ask, &quot;How are you sure of that?&quot;, &quot;How are you sure it's not just a memory leak?&quot;, Well.</p>
  <p><a href="https://i.stack.imgur.com/QXrsy.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/QXrsy.png" alt="enter image description here" /></a></p>
  <p>This test run much longer. I run <code>malloc_trim(0)</code>, whenever possible in my program. And it seems, application can finally return the unused memory to the OS, and it goes almost to zero (the actual data size my program has currently). This implies the problem is not a memory leak. But I can't rely on this behavior, the allocation and freeing pattern of my program is random, what if it never releases the memory ?</p>
  <ul>
  <li>I said memory pools are a bad idea for this project in the title. Of course I don't have absolute knowledge. But the strings I am allocating can be anything between 30-4000 bytes. Which makes many optimizations and clever ideas much harder. Memory pools are one of them.</li>
  <li>I am using <code>GCC 11 / G++ 11</code> as a compiler. If some old versions have bad allocators. I shouldn't have that problem.</li>
  <li>How am I getting memory usage ? Python <code>psutil</code> module. <code>proc.memory_full_info()[0]</code>, which gives me <code>RSS</code>.</li>
  <li>Of course, you don't know the details of my program. It is still a valid question, if this is indeed because of heap fragmentation. Well what I can say is, I am keeping a <em>up to date</em> information about how many allocations and frees took place. And I know the element counts of every container in my program. But if you still have some ideas about the causes of the problem, I am open to suggestions.</li>
  <li>I can't just allocate, say 4096 bytes for all the strings so it would become easier to optimize. That's the opposite I am trying to do.</li>
  </ul>
  <p>So my question is, what do programmers do(what should I do), in an application where millions of alloc's and free's take place over time, and they are of different sizes so memory pools are hard to use efficiently. I can't change what the program does, I can only change implementation details.</p>
  <p><strong>Bounty Edit:</strong> When trying to utilize memory pools, isn't it possible to make multiple of them, to the extent that there is a pool for every possible byte count ? For example my strings can be something in between 30-4000 bytes. So couldn't somebody make <code>4000 - 30 + 1</code>, 3971 memory pools, for each and every possible allocation size of the program. Isn't this applicable ? All pools could start small (no not lose much memory), then enlarge, in a balance between performance and memory. I am not trying to make a use of memory pool's ability to reserve big spaces beforehand. I am just trying to effectively reuse freed space, because of frequent alloc's and free's.</p>
  <p><strong>Last edit:</strong> It turns out that, the memory growth appearing in the graphs, was actually from a http request queue in my program. I failed to see that hundreds of thousands of tests that I did, bloated this queue (something like webhook). And the reasonable explanation of figure 2 is, I finally get DDOS banned from the server (or can't open a connection anymore for some reason), the queue emptied, and the RAM issue resolved. So anyone reading this question later in the future, consider every possibility. It would have never crossed my mind that it was something like this. Not a memory leak, but an implementation detail. Still I think @Hajo Kirchhoff deserves the bounty, his answer was really enlightening.</p>