title: Performance of Redis vs Disk in caching application
body: <p>I wanted to create a redis cache in python, and as any self respecting scientist I made a bench mark to test the performance.</p>
  
  <p>Interestingly, redis did not fare so well. Either Python is doing something magic (storing the file) or my version of redis is stupendously slow.</p>
  
  <p>I don't know if this is because of the way my code is structured, or what, but I was expecting redis to do better than it did.</p>
  
  <p>To make a redis cache, I set my binary data (in this case, an HTML page) to a key derived from the filename with an expiration of 5 minutes. </p>
  
  <p>In all cases, file handling is done with f.read() (this is ~3x faster than f.readlines(), and I need the binary blob). </p>
  
  <p>Is there something I'm missing in my comparison, or is Redis really no match for a disk? Is Python caching the file somewhere, and reaccessing it every time? Why is this so much faster than access to redis? </p>
  
  <p>I'm using redis 2.8, python 2.7, and redis-py, all on a 64 bit Ubuntu system.</p>
  
  <p>I do not think Python is doing anything particularly magical, as I made a function that stored the file data in a python object and yielded it forever. </p>
  
  <p>I have four function calls that I grouped:</p>
  
  <p>Reading the file X times</p>
  
  <p>A function that is called to see if redis object is still in memory, load it, or cache new file (single and multiple redis instances).</p>
  
  <p>A function that creates a generator that yields the result from the redis database (with single and multi instances of redis).</p>
  
  <p>and finally, storing the file in memory and yielding it forever.</p>
  
  <pre><code>import redis
  import time
  
  def load_file(fp, fpKey, r, expiry):
      with open(fp, "rb") as f:
          data = f.read()
      p = r.pipeline()
      p.set(fpKey, data)
      p.expire(fpKey, expiry)
      p.execute()
      return data
  
  def cache_or_get_gen(fp, expiry=300, r=redis.Redis(db=5)):
      fpKey = "cached:"+fp
  
      while True:
          yield load_file(fp, fpKey, r, expiry)
          t = time.time()
          while time.time() - t - expiry &lt; 0:
              yield r.get(fpKey)
  
  
  def cache_or_get(fp, expiry=300, r=redis.Redis(db=5)):
  
      fpKey = "cached:"+fp
  
      if r.exists(fpKey):
          return r.get(fpKey)
  
      else:
          with open(fp, "rb") as f:
              data = f.read()
          p = r.pipeline()
          p.set(fpKey, data)
          p.expire(fpKey, expiry)
          p.execute()
          return data
  
  def mem_cache(fp):
      with open(fp, "rb") as f:
          data = f.readlines()
      while True:
          yield data
  
  def stressTest(fp, trials = 10000):
  
      # Read the file x number of times
      a = time.time()
      for x in range(trials):
          with open(fp, "rb") as f:
              data = f.read()
      b = time.time()
      readAvg = trials/(b-a)
  
  
      # Generator version
  
      # Read the file, cache it, read it with a new instance each time
      a = time.time()
      gen = cache_or_get_gen(fp)
      for x in range(trials):
          data = next(gen)
      b = time.time()
      cachedAvgGen = trials/(b-a)
  
      # Read file, cache it, pass in redis instance each time
      a = time.time()
      r = redis.Redis(db=6)
      gen = cache_or_get_gen(fp, r=r)
      for x in range(trials):
          data = next(gen)
      b = time.time()
      inCachedAvgGen = trials/(b-a)
  
  
      # Non generator version    
  
      # Read the file, cache it, read it with a new instance each time
      a = time.time()
      for x in range(trials):
          data = cache_or_get(fp)
      b = time.time()
      cachedAvg = trials/(b-a)
  
      # Read file, cache it, pass in redis instance each time
      a = time.time()
      r = redis.Redis(db=6)
      for x in range(trials):
          data = cache_or_get(fp, r=r)
      b = time.time()
      inCachedAvg = trials/(b-a)
  
      # Read file, cache it in python object
      a = time.time()
      for x in range(trials):
          data = mem_cache(fp)
      b = time.time()
      memCachedAvg = trials/(b-a)
  
  
      print "\n%s file reads: %.2f reads/second\n" %(trials, readAvg)
      print "Yielding from generators for data:"
      print "multi redis instance: %.2f reads/second (%.2f percent)" %(cachedAvgGen, (100*(cachedAvgGen-readAvg)/(readAvg)))
      print "single redis instance: %.2f reads/second (%.2f percent)" %(inCachedAvgGen, (100*(inCachedAvgGen-readAvg)/(readAvg)))
      print "Function calls to get data:"
      print "multi redis instance: %.2f reads/second (%.2f percent)" %(cachedAvg, (100*(cachedAvg-readAvg)/(readAvg)))
      print "single redis instance: %.2f reads/second (%.2f percent)" %(inCachedAvg, (100*(inCachedAvg-readAvg)/(readAvg)))
      print "python cached object: %.2f reads/second (%.2f percent)" %(memCachedAvg, (100*(memCachedAvg-readAvg)/(readAvg)))
  
  if __name__ == "__main__":
      fileToRead = "templates/index.html"
  
      stressTest(fileToRead)
  </code></pre>
  
  <p>And now the results:</p>
  
  <pre><code>10000 file reads: 30971.94 reads/second
  
  Yielding from generators for data:
  multi redis instance: 8489.28 reads/second (-72.59 percent)
  single redis instance: 8801.73 reads/second (-71.58 percent)
  Function calls to get data:
  multi redis instance: 5396.81 reads/second (-82.58 percent)
  single redis instance: 5419.19 reads/second (-82.50 percent)
  python cached object: 1522765.03 reads/second (4816.60 percent)
  </code></pre>
  
  <p>The results are interesting in that a) generators are faster than calling functions each time, b) redis is slower than reading from the disk, and c) reading from python objects is ridiculously fast.</p>
  
  <p>Why would reading from a disk be so much faster than reading from an in-memory file from redis?</p>
  
  <p>EDIT:
  Some more information and tests.</p>
  
  <p>I replaced the function to </p>
  
  <pre><code>data = r.get(fpKey)
  if data:
      return r.get(fpKey)
  </code></pre>
  
  <p>The results do not differ much from </p>
  
  <pre><code>if r.exists(fpKey):
      data = r.get(fpKey)
  
  
  Function calls to get data using r.exists as test
  multi redis instance: 5320.51 reads/second (-82.34 percent)
  single redis instance: 5308.33 reads/second (-82.38 percent)
  python cached object: 1494123.68 reads/second (5348.17 percent)
  
  
  Function calls to get data using if data as test
  multi redis instance: 8540.91 reads/second (-71.25 percent)
  single redis instance: 7888.24 reads/second (-73.45 percent)
  python cached object: 1520226.17 reads/second (5132.01 percent)
  </code></pre>
  
  <p>Creating a new redis instance on each function call actually does not have a noticable affect on read speed, the variability from test to test is larger than the gain.</p>
  
  <p>Sripathi Krishnan suggested implementing random file reads. This is where caching starts to really help, as we can see from these results.</p>
  
  <pre><code>Total number of files: 700
  
  10000 file reads: 274.28 reads/second
  
  Yielding from generators for data:
  multi redis instance: 15393.30 reads/second (5512.32 percent)
  single redis instance: 13228.62 reads/second (4723.09 percent)
  Function calls to get data:
  multi redis instance: 11213.54 reads/second (3988.40 percent)
  single redis instance: 14420.15 reads/second (5157.52 percent)
  python cached object: 607649.98 reads/second (221446.26 percent)
  </code></pre>
  
  <p>There is a HUGE amount of variability in file reads so the percent difference is not a good indicator of speedup.</p>
  
  <pre><code>Total number of files: 700
  
  40000 file reads: 1168.23 reads/second
  
  Yielding from generators for data:
  multi redis instance: 14900.80 reads/second (1175.50 percent)
  single redis instance: 14318.28 reads/second (1125.64 percent)
  Function calls to get data:
  multi redis instance: 13563.36 reads/second (1061.02 percent)
  single redis instance: 13486.05 reads/second (1054.40 percent)
  python cached object: 587785.35 reads/second (50214.25 percent)
  </code></pre>
  
  <p>I used random.choice(fileList) to randomly select a new file on each pass through the functions. </p>
  
  <p>The full gist is here if anyone would like to try it out - <a href="https://gist.github.com/3885957" rel="noreferrer">https://gist.github.com/3885957</a></p>
  
  <p>Edit edit:
  Did not realize that I was calling one single file for the generators (although the performance of the function call and generator was very similar). Here is the result of different files from the generator as well.</p>
  
  <pre><code>Total number of files: 700
  10000 file reads: 284.48 reads/second
  
  Yielding from generators for data:
  single redis instance: 11627.56 reads/second (3987.36 percent)
  
  Function calls to get data:
  single redis instance: 14615.83 reads/second (5037.81 percent)
  
  python cached object: 580285.56 reads/second (203884.21 percent)
  </code></pre>