title: Sidekiq not deallocating memory after workers have finished
body: <p>I have about <strong>six Sidekiq worker</strong> which perform <strong>JSON crawling</strong>. Dependent on the endpoint's dataset size they finish between 1min and 4h. Especially, watching the long one, which takes 4h, I see a very slight increase of memory over time.</p>
  
  <p>It's not a problem, until I want to schedule the same worker jobs again. The memory is not deallocated and stacks up, until I run into the Linux OOM Killer which gets rid of my Sidekiq process.</p>
  
  <p><strong>Memory leak</strong>? I watched the number of different objects in ObjectSpace:</p>
  
  <pre><code>ObjectSpace.each_object.inject(Hash.new(0)) { |count, o| count[o.class] += 1 }
  </code></pre>
  
  <p>There is not really an increase there, the set of hashes, arrays, etc. stays the same, short increases are swept away by the Garbage Collector and <code>gc.stat[:count]</code> tells me, that the Garbage Collector is working, too.</p>
  
  <p>Even after the worker finishes, e.g. I get the <strong>[Done]</strong> logged and no workers are busy any more, the memory is not deallocated. What are the reasons for that? Can I do something against this? Write a finalizer?</p>
  
  <p>The only <strong>current solution</strong>: Restart the Sidekiq process.</p>
  
  <p>I am on Ruby 2.0.0 and use Ruby MRI.</p>
  
  <hr>
  
  <p>For the JSON parsing I use <a href="https://github.com/brianmario/yajl-ruby" rel="noreferrer">Yajl</a>, thus a C binding. I need it because it seems the only fast JSON parser that properly implements streamed reading and writing.</p>