How do you kill Futures once they have started?
Question: I am using the new
[`concurrent.futures`](https://docs.python.org/3/library/concurrent.futures.html)
module (which also has a Python 2 backport) to do some simple multithreaded
I/O. I am having trouble understanding how to cleanly kill tasks started using
this module.

Check out the following Python 2/3 script, which reproduces the behavior I'm
seeing:

    
    
    #!/usr/bin/env python
    from __future__ import print_function
    
    import concurrent.futures
    import time
    
    
    def control_c_this():
        with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
            future1 = executor.submit(wait_a_bit, name="Jack")
            future2 = executor.submit(wait_a_bit, name="Jill")
    
            for future in concurrent.futures.as_completed([future1, future2]):
                future.result()
    
            print("All done!")
    
    
    def wait_a_bit(name):
        print("{n} is waiting...".format(n=name))
        time.sleep(100)
    
    
    if __name__ == "__main__":
        control_c_this()
    

While this script is running it appears impossible to kill cleanly using the
regular Control-C keyboard interrupt. I am running on OS X.

  * On Python 2.7 I have to resort to `kill` from the command line to kill the script. Control-C is just ignored.
  * On Python 3.4, Control-C works if you hit it twice, but then a lot of strange stack traces are dumped.

Most documentation I've found online talks about how to cleanly kill threads
with the old `threading` module. None of it seems to apply here.

And all the methods provided within the `concurrent.futures` module to stop
stuff (like `Executor.shutdown()` and `Future.cancel()`) only work when the
Futures haven't started yet or are complete, which is pointless in this case.
I want to interrupt the Future immediately.

My use case is simple: When the user hits Control-C, the script should exit
immediately like any well-behaved script does. That's all I want.

So what's the proper way to get this behavior when using `concurrent.futures`?


Answer: It's kind of painful. Essentially, your worker threads have to be finished
before your main thread can exit. You cannot exit unless they do. The typical
workaround is to have some global state, that each thread can check to
determine if they should do more work or not.

Here's the
[quote](https://hg.python.org/cpython/file/2805b0dca798/Lib/concurrent/futures/thread.py#l15)
explaining why. In essence, if threads exited when the interpreter does, bad
things could happen.

Here's a working example. Note that C-c takes at most 1 sec to propagate
because the sleep duration of the child thread.

    
    
    #!/usr/bin/env python
    from __future__ import print_function
    
    import concurrent.futures
    import time
    import sys
    
    quit = False
    def wait_a_bit(name):
        while not quit:
            print("{n} is doing work...".format(n=name))
            time.sleep(1)
    
    def setup():
        executor = concurrent.futures.ThreadPoolExecutor(max_workers=5)
        future1 = executor.submit(wait_a_bit, "Jack")
        future2 = executor.submit(wait_a_bit, "Jill")
    
        # main thread must be doing "work" to be able to catch a Ctrl+C 
        # http://www.luke.maurits.id.au/blog/post/threads-and-signals-in-python.html
        while (not (future1.done() and future2.done())):
            time.sleep(1)
    
    if __name__ == "__main__":
        try:
            setup()
        except KeyboardInterrupt:
            quit = True
    

