title: stackexchange redis, reading large file = timeouts
body: <p>I have an application that uses a large in memory file (just under 2gb).</p>
  
  <p>I'm trying to use redis lists (in Azure) as the storage (vs SQL). Building the list in redis is pretty fast, I can load the redis list in about 5 minutes but then I need to read from the list into the application. </p>
  
  <p>This is extremely slow, I've tried increasing the threads, extending synctimeout etc to no avail.</p>
  
  <pre><code>ThreadPool.SetMinThreads(200, 200);   
  </code></pre>
  
  <p>I'm using a C# implementation of redis list that I discovered online, I'm passing this to the code that builds the in memory collection via foreach loop. Internally this is how it processes the data (I've omitted the rest of the class)</p>
  
  <pre><code>    public class RedisList&lt;T&gt; : IList&lt;T&gt;
  {
      private static ConnectionMultiplexer _cnn;
      private readonly string _key;
  
      private static readonly Lazy&lt;ConnectionMultiplexer&gt; LazyConnection = new Lazy&lt;ConnectionMultiplexer&gt;(() =&gt; ConnectionMultiplexer.Connect(ConfigurationManager.AppSettings["AzureRedisCacheUrl"]));
  
      public ConnectionMultiplexer Connection =&gt; LazyConnection.Value;
      public RedisList(string key)
      {
          this._key = key;
          _cnn = Connection;
      }
  
      public IEnumerator&lt;T&gt; GetEnumerator()
      {
          for (var i = 0; i &lt; this.Count; i++)
          {
              yield return Deserialize&lt;T&gt;(GetRedisDb().ListGetByIndex(_key, i).ToString());
          }
      }
  }
  </code></pre>
  
  <p>Is there a more efficient way of reading in the data? Am I insane to do it this way? :D Thanks</p>