title: Filtering Redis Hash Entries
body: <p>I'm using redis to store hashes with ~100k records per hash. I want to implement filtering (faceting) the records within a given hash. Note a hash entry can belong to n filters.</p>
  
  <p>After reading <a href="http://playnice.ly/blog/2010/05/24/redis-multi-field-searching-and-filtering/">this</a> and <a href="http://rediscookbook.org/pipeline_multiple_commands.html">this</a> it looks like I should:</p>
  
  <ol>
  <li>Implement a sorted SET per filter. The values within the SET correspond to the keys within a HASH.</li>
  <li>Retrieve the HASH keys from the given filter SET.</li>
  <li>Once I have the HASH keys from the SET fetch the corresponding entries from the HASH. This should give me all entries that belong to the filter.</li>
  </ol>
  
  <p>Firstly is the above approach correct at a high level?</p>
  
  <p>Assuming the approach is OK the bit I'm missing is what's the most efficient implementation to retrieve the HASH entries? Am I right in thinking once I have the HASH keys I should then use a PIPELINE to queue multiple HGETALL commands passing through each HASH key? Is there a better approach?</p>
  
  <p>My concern about using a PIPELINE is that I believe it will block all other clients while servicing the command. I'll be paging the filtered results with 500 results per page. With multiple browser based clients performing filtering, not to mention the back end processes that populate the SETs and HASHes it sounds like there's potential for a lot of contention if PIPELINE does block. Could anyone provide a view on this?</p>
  
  <p>If it helps I'm using 2.2.4 redis, predis for the web clients and servicestack for the back end.</p>
  
  <p>Thanks,
  Paul</p>