title: SQLite3 - memory usage gradually increases until all available memory are consumed
body: <p>I am experiencing a memory usage issue while running a simple single-threaded C++ program for long time (say over night). The program uses SQLite3 API to open a data base and writes some data in it within a loop.I am running the program on two different machines: a desktop Ubuntu Linux and an ARM based embedded device running a custom built Linux.
  In both cases I am getting the same results: Memory is consumed gradually and not released while application is running. I am checking memory usage using a simple bash script running in background:</p>
  
  <pre><code>while true;
  do free -m;
  sleep 2;
  done
  </code></pre>
  
  <p>It should be noted that I am also monitoring memory usage using SQLite provided API:</p>
  
  <pre><code>sqlite3_memory_used() 
  </code></pre>
  
  <p>The API reports a fairly stable amount of used memory but the "free -m" report is different and increases gradually.</p>
  
  <p>The SQLite source code is compiled with the following flags:</p>
  
  <pre><code>SQLITE_DEFAULT_TEMP_CACHE_SIZE=3
  SQLITE_DEFAULT_WAL_AUTOCHECKPOINT=2
  SQLITE_MAX_MMAP_SIZE=2048
  SQLITE_ENABLE_MEMSYS5
  SQLITE_ENABLE_MEMORY_MANAGEMENT
  SQLITE_DEFAULT_CACHE_SIZE=3
  SQLITE_DEFAULT_AUTOVACUUM=1
  SQLITE_DEFAULT_PAGE_SIZE=512
  </code></pre>
  
  <p>Please note that at this stage I am not concerned about speed but my main concern is memory usage so I set parameters in such a way that minimum data is cached in memory and get them pushed to the disk as soon as possible.</p>
  
  <p>I also use "PRAGMA shrink_memory" in each iteration. </p>
  
  <p>To minimise dynamic memory allocation I have also provided static arrays for the following memory types:</p>
  
  <pre><code>SQLITE_CONFIG_HEAP
  SQLITE_CONFIG_SCRATCH
  SQLITE_CONFIG_PAGECACHE
  </code></pre>
  
  <p>And the code snippet which writes to database looks like this:</p>
  
  <pre><code>char SQL_Statement[100]={0};
  char *ErrMsg = 0;
  for (int i = 0; i &lt; 1000000; i++)
  {
      sprintf(SQL_Statement, "INSERT INTO PointValue (TimeStamp, BlockId, PointId, Value) VALUES (%f, %d, %d, %d);",TimeStamp_ ,BlockId_, PointId_, Value_ );
      check =  sqlite3_exec(MyDB, SQL_Statement, callback, (void*)data, &amp;ErrMsg); 
      sqlite3_free(ErrMsg);
  }
  </code></pre>