Pyspark Handle Ctrl + Z?
Question: When running pyspark, if you do a `Ctrl` \+ `Z` in the middle of a process, it
stops the job from running but does not close SparkContext. This means the
port is not closed either and spark has to jump through several ports to
connect the next time someone runs a spark job. I want to prevent people from
using `Ctrl` \+ `Z` while running these scripts, but the standard python
handler doesn't seem to work when SparkContext is running, which I assume is a
result of the fact that it's technically running in java and I've only set a
handler for python? Here is a sample of what I've written.

    
    
        import signal
        from pyspark import SparkContext
        sc = SparkContext()
    
        def handler(signum, frame):
            print 'Please refrain from using CTRL + Z, use CTRL + C instead'
    
        signal.signal(signal.SIGTSTP, handler)
    
        while True:
            pass
    

When I run this code, pressing `Ctrl` \+ `Z` still does the default behavior
of pausing rather than executing my handler function.

This code does work the way I want it to if I eliminate the spark content as
so:

    
    
        import signal
    
        def handler(signum, frame):
            print 'Please refrain from using CTRL + Z, use CTRL + C instead'
    
        signal.signal(signal.SIGTSTP, handler)
    
        while True:
            pass
    


Answer: What I ended up doing was adding `trap "" SIGTSTP` to my spark-submit shell
script just before the spark-submit.

