title: How to prevent deadlock in try/except/finally while interrupting the main method from a thread
body: <p>I saw this deadlock in an application which resulted in it getting stuck. The application has a main thread as well as a thread that processes some work in the background.</p>
  
  <p>A connection error occurred to a service both threads use at the same time. The main thread catches exceptions with <code>except Exception</code>, the background thread catches errors and interrupts the main application whenever they occur. A deadlock occurs in the python logging module while acquiring the lock to serialize writes to standard output. Below is a reproduction of the deadlock (without logging modulo, but mimics its behavior)</p>
  
  <pre><code>import thread
  from threading import Thread, RLock
  
  lock = RLock()
  
  def log(msg):
      lock.acquire()
      try:
          print msg
      finally:
          lock.release()
  
  def run():
      thread.interrupt_main()
  
  Thread(target=run).start()
  try:
      1 / 0
  except KeyboardInterrupt:
      log('keyboardinterrupt')
  except Exception, e:
      log('Exception')
      raise
  finally:
      log('finally')
  </code></pre>
  
  <p>Run it in a loop and the deadlock will eventually occur.</p>
  
  <pre><code>while true; do python deadlock.py; done
  </code></pre>
  
  <p><code>finally</code> and <code>except</code> execution kick off together from what I see in the docs, both ask for the lock, one of them wins, then the application is interrupted by the thread, and if that happens before the owning thread releases the lock, the other thread is left waiting for the lock and the application gets stuck doing nothing.</p>
  
  <p>A fix would be to acquire the lock in a try and release it in a finally, but it is acquired outside the try in python <a href="https://github.com/python/cpython/blob/2.7/Lib/logging/__init__.py#L757" rel="nofollow noreferrer">code</a>. Which I think makes sense since it is supposed to block until it gets the lock or fail.</p>
  
  <p>I can do that in my application with something like:</p>
  
  <pre><code>try:
      log.info('msg')
  finally:
      for handler in log.handlers:  # (and its parents' handlers)
          try:
              handler.lock.release()
          except:  # ignore if it was not acquired
              pass
  </code></pre>
  
  <p>But that does not look right, and I am wondering if there is a better solution here?</p>
  
  <p>I thought about not interrupting the main thread when an error occurs in the thread and instead use an error queue (similar to go), and check the queue in the main app loop and raise an error from there, but that can add delay to surfacing these errors.</p>