title: Redis 1+ min query time on lists with 15 larger JSON objects (20MB total)
body: <p>I use Redis to cache database inserts. For this I created a list <code>CACHE</code> into which I push serialized JSON lists. In pseudocode:</p>
  
  <pre><code>let entries = [{a}, {b}, {c}, ...];
  redis.rpush("CACHE", JSON.stringify(entries));
  </code></pre>
  
  <p>The idea is to run this code for an hour, then later do an</p>
  
  <pre><code>let all = redis.lrange("CACHE", 0, LIMIT);
  processAndInsert(all);
  redis.ltrim("CACHE", 0, all.length);
  </code></pre>
  
  <p>Now the thing is that each <code>entries</code> can be relatively large (but far below 512MB / whatever Redis limit I read about). Each of the <code>a, b, c</code> is an object of probably 20 bytes, and <code>entries</code> itself can easily have 100k+ objects / 2MB.</p>
  
  <p>My problem now is that even for very short <code>CACHE</code> lists of only 15 entries a simple <code>lrange</code> can take many minutes(!) even from the <code>redis-cli</code> (my <code>node.js</code> actually dies with an <em>"FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory"</em>, but that's a side comment).</p>
  
  <p>The debug output for the list looks like this:</p>
  
  <pre><code>127.0.0.1:6379&gt; debug object "CACHE"
  Value at:00007FF202F4E330 refcount:1 encoding:linkedlist serializedlength:18104464 lru:12984004 lru_seconds_idle:1078
  </code></pre>
  
  <p>What is happening? Why is this so massively slow, and what can I do about it? This does not seem to be a <em>normal slowness</em>, something seems to be fundamentally wrong.</p>
  
  <p>I am using a local Redis 2.8.2101 (x64), ioredis 1.6.1, node.js 0.12 on a relatively hardcore Windows 10 gaming machine (i5, 16GB RAM, 840 EVO SSD, ...) by the way.</p>