title: Debugging a Python script that randomly hangs and uses 100% of a processor core
body: <p>I'm currently working on a fairly complex multithreaded Python script. There is one main function that is run in about 5 threads at a time. I've been having some issues with it hanging and using 100% of the processor core that it is running on. This hanging occurs after the main function has been run hundreds of times, so it's hard to pinpoint exactly when or where it is happening. Once the program hangs, it never starts running again.</p>
  
  <p>It seems that it is only one thread that hangs at a time, so I didn't really understand why it was hanging the entire program. That's when I found <a href="https://stackoverflow.com/a/11265981/1289152">this Stack Overflow solution</a> that explained, "In some Python implementations, only one Python thread can execute at a time. Threads in CPython are only really useful for multiplexing IO operations, not for putting CPU-intensive tasks in the background." So when one thread hangs with full CPU usage, the entire program understandably comes to a halt.</p>
  
  <p>Below is a screenshot of Process Explorer's view of the python.exe process when the program has hung. As you can see, only one thread is actually doing something.</p>
  
  <p><img src="https://i.stack.imgur.com/1CPMe.png" alt="Process Manager screenshot"></p>
  
  <p>I'd like to be able to analyze exactly what lines were executed before the script hung. I don't really know where I could insert a breakpoint using "import pdb; pdb.set_trace()" because I don't know when or where it'll screw up. I can't manually step through the program since it takes 30 minutes to a few hours of running for it to hang. I tried looking through my script to find any obvious infinite loops that could result or anything like that, but I can't seem to figure out what is causing the hang.</p>
  
  <p>My question is this: how would I go about debugging this? I'd ideally just like to see what lines were executed right before it hung, but I don't even know how to detect when it hangs. I can't post the full script here, so hopefully someone will know how I can debug this. Thanks in advance.</p>