title: Trying to add throttle control to paralleled API calls in python
body: <p>I am using Google places API which has a <strong>query per second</strong> limit of <strong>10</strong>. This means I cannot make more than 10 requests within a second. If we were using Serial execution this wouldn't be an issue as the APIs avg response time is 250 ms, so i will be able to make just 4 calls in a second.</p>
  
  <p>To utilize the entire <strong>10 QPS limit</strong> i used <strong>multithreading</strong> and made parallel API calls. But now i need to control the number of calls that can happen in a second, it should not go beyond <strong>10</strong> (google API starts throwing errors if i cross the limit)</p>
  
  <p>Below is the code that i have so far, I am not able to figure out why the program just gets stuck sometimes or takes alot longer than required.</p>
  
  <pre><code>import time
  from datetime import datetime
  import random
  from threading import Lock
  from concurrent.futures import ThreadPoolExecutor as pool
  import concurrent.futures
  import requests
  import matplotlib.pyplot as plt
  from statistics import mean
  from ratelimiter import RateLimiter
  
  def make_parallel(func, qps=10):
      lock = Lock()
      threads_execution_que = []
      limit_hit = False
      def qps_manager(arg):
          current_second = time.time()
          lock.acquire()
          if len(threads_execution_que) &gt;= qps or limit_hit:
              limit_hit = True
              if current_second - threads_execution_que[0] &lt;= 1:
                  time.sleep(current_second - threads_execution_que[0])
          current_time = time.time()
          threads_execution_que.append(current_time)
          lock.release()
  
          res = func(arg)
  
          lock.acquire()
          threads_execution_que.remove(current_time)
          lock.release()
          return res
  
      def wrapper(iterable, number_of_workers=12):
          result = []
          with pool(max_workers=number_of_workers) as executer:
              bag = {executer.submit(func, i): i for i in iterable}
              for future in concurrent.futures.as_completed(bag):
                  result.append(future.result())
          return result
      return wrapper
  
  @make_parallel
  def api_call(i):
      min_func_time = random.uniform(.25, .3)
      start_time = time.time()
      try:
          response = requests.get('https://jsonplaceholder.typicode.com/posts', timeout=1)
      except Exception as e:
          response = e
      if (time.time() - start_time) - min_func_time &lt; 0:
          time.sleep(min_func_time - (time.time() - start_time))
      return response
  
  api_call([1]*50)
  </code></pre>
  
  <p>Ideally the code should take not more than 1.5 seconds, but currently it is taking about 12-14 seconds.
  <strong>The script speeds up to its expected speed as soon as i remove the QPS manager logic.</strong></p>
  
  <p>Please do suggest what i am doing wrong and also, if there is any package available already which does this mechanism out of the box.</p>