title: Fastest pythonic way of parsing dictionary where values are bytes stringfied json object
body: <p>So I have a dictionary which is a hash object I'm getting from Redis, similar to the following dictionary:</p>
  
  <pre><code>source_data = {
     b'key-1': b'{"age":33,"gender":"Male"}', 
     b'key-2': b'{"age":20,"gender":"Female"}'
  }
  </code></pre>
  
  <p>My goal is extract all the values from this dictionary and have them as a list of Python dictionaries like so:</p>
  
  <pre><code>final_data = [
     {
        'age': 33,
        'gender': 'Male'
     },
  
     {
        'age': 20,
        'gender': 'Female'
     }
  ]
  </code></pre>
  
  <p>I tried list comprehension with json parsing:</p>
  
  <pre><code>import json
  final_data = [json.loads(a) for a in source_data.values()]
  </code></pre>
  
  <p>It works but for large data set, it takes too much time.</p>
  
  <p>I switch to using this 3rd party json module <a href="https://pypi.org/project/ujson/" rel="nofollow noreferrer">ujson</a> which is faster according to this <a href="https://artem.krylysov.com/blog/2015/09/29/benchmark-python-json-libraries/" rel="nofollow noreferrer">benchmark</a>, but I haven't noticed any improvement.</p>
  
  <p>I tried using multi-threading :</p>
  
  <pre><code>pool = Pool()
  final_data = pool.map(ujson.loads, source_data.values(), chunksize=500)
  
  pool.close()
  pool.join()
  </code></pre>
  
  <p>I played a bit with <code>chunksize</code> but the result is the same, still taking too much time.</p>
  
  <p>It would be super helpful if someone can suggest another solution or improvement to previous tries, it would be ideal if I could avoid using a loop.</p>