Scikit Learn - Diagnosing when parallel jobs get stuck
I was unable to definitively find the cause of this problem, but it stopped happening when I increased the amount of memory available.  So it seems reasonable to conclude that one of the children processes encountered a MemoryError and just died.
What is the proper way to diagnose what is happening when parallel jobs get stuck in Scikit-Learn?
Specifically, I have had several jobs that appear to finish (htop shows no CPU activity), but python stops responding.  Pressing Ctrl+c doesn't exit (though it does register a KeyboardInterrupt, it doesn't kill the python process), and the process must be killed from shell.  Total memory usage approaches the capacity of the machine, but I get no explicit errors that there was a MemoryError.
This has occurred with RandomForestRegressor, and also with cross_validation.cross_val_score, under both 0.14 and master on Ubuntu/Debian.
I suspect this is a memory issue, since the jobs seem to complete without a problem on machines with more memory.