title: Python Threads - Changing number of parallel threads on the fly
body: <p>I am trying out concurrent.futures.ThreadPoolExecutor to run a few jobs in parallel. It lets me set the max_workers once, at the initialization time. But for some reason, my requirement is to change the number of active threads on the fly. I couldn't find any way to do that. Tried resetting the "_max_workers" on the fly, but that had no impact.</p>
  
  <p>Here's the code.</p>
  
  <pre><code>from concurrent.futures import ThreadPoolExecutor, as_completed
  import urllib.request
  from random import randint
  from time import sleep
  
  URLS = ['http://www.foxnews.com/',
          'http://www.cnn.com/',
          'http://europe.wsj.com/',
          'http://www.bbc.co.uk/',
          'http://some-made-up-domain.com/']
  
  # Retrieve a single page and report the url and contents
  def load_url(url, timeout):
      with urllib.request.urlopen(url, timeout=timeout) as conn:
          return conn.read()
  
  if __name__ == "__main__":
      pool = ThreadPoolExecutor(1)
      fs = []
      for url in URLS:
          fs.append(pool.submit(load_url, url, 60))
  
      r = randint(1,5)
      print ("Changing max_workers to: {}".format(r))
      pool._max_workers = r
      print ("Current queue size: {}. Number of threads in parallel: {}".format(pool._work_queue.qsize(), len(pool._threads)))
  
      for f in as_completed(fs):
          r = randint(1,5)
          print ("Changing max_workers to: {}".format(r))
          pool._max_workers = r
          print ("Current queue size: {}. Number of threads in parallel: {}".format(pool._work_queue.qsize(), len(pool._threads)))
  
          try:
              data = f.result()
          except Exception as exc:
              print('%r generated an exception: %s' % (url, exc))
          else:
              print('Page is %d bytes' % (len(data)))
  
      print ("done")
  </code></pre>
  
  <p>Here's what it prints:</p>
  
  <pre><code>Changing max_workers to: 3  
  Current queue size: 4. Number of threads in parallel: 1  
  Changing max_workers to: 1  
  Current queue size: 3. Number of threads in parallel: 1  
  Page is 233315 bytes  
  Changing max_workers to: 4  
  Current queue size: 2. Number of threads in parallel: 1  
  Page is 1127911 bytes  
  Changing max_workers to: 2  
  Current queue size: 1. Number of threads in parallel: 1  
  Page is 2741410 bytes  
  Changing max_workers to: 4  
  Current queue size: 0. Number of threads in parallel: 1  
  Page is 296361 bytes  
  done  
  </code></pre>
  
  <p>Is there a way I can change the max_worker which will change the active threads in pool._threads?</p>