title: Releasing Memory After Threads End
body: <p>I'm working on an application that is primarily an API, but also has a multithreaded background job processing system, used to perform scheduled jobs as well as ad hoc jobs that take too long for an instant API response.</p>
  
  <p>This will be forked 10 times through gunicorn. Any single forked process is capable of picking up a job to run, so job processing is balanced between the processes alongside the servicing of API requests.</p>
  
  <p>My challenge is with how each process will continue to claim the peak amount of memory it ever needed for job processing. Some jobs require 1.5GB-2GB worth of memory.</p>
  
  <p>Given enough time, eventually all 10 processes will have had to work those kinds of jobs, and each will be clinging to upwards of 2GB of memory. Even if the average memory use of the process rarely exceeds 100MB.</p>
  
  <p>These intensive jobs are only run through dedicated threads within the process.</p>
  
  <p>Is there ANY mechanism to compel Python to release the memory claimed specifically for a thread at the thread's closure? Or any general mechanism to force a Python process to reset memory to only what's actively needed at that moment?</p>
  
  <p>Side note: I'm also exploring forking instead of threading but thus far that's introducing other problems I'm not sure I can work around.</p>