Processes don't terminate when I call Python script using crontab
Question: I have a script called yarn_monitor.py. When I run it, the program executes
correctly and when I look at the running processes using `ps -u myname`,
everything is clear.

But when I run yarn_monitor.py using cron:

    
    
    * * * * * /home/me/projects/yarn_monitor/yarn_monitor.py
    

I see several processes that don't quit. Here they are repeated twice:

    
    
    19337 ?        00:00:00 yarn_monitor.py
    19338 ?        00:00:03 java
    19418 ?        00:00:00 sendmail
    19419 ?        00:00:00 postdrop
    20043 ?        00:00:00 yarn_monitor.py
    20046 ?        00:00:02 java
    20199 ?        00:00:00 sendmail
    20200 ?        00:00:00 postdrop
    

Eventually, as I let cron keep running the job, I get a Java GC out of memory
error.

As far as I know I'm not using any Java in my process. Here are my imports:

    
    
    from __future__ import print_function
    import os
    import sys
    import re
    import time
    from pysnmp.entity.rfc3413.oneliner import ntforg
    from pysnmp.proto import rfc1902
    

Any idea what could be happening? Or ways to keep running this job while
killing previously created processes?


Answer: > It looks like the close isn't working correctly when run through cron, but
> it works fine when called manually.

Sounds to me like yarn is expecting some environment variables to be set which
aren't set when run as a cron job. Try the following to debug this:

    
    
    * * * * * yarn application -list > /home/me/error_log 2>&1 
    

Now wait 1 minute and look into `/home/me/error_log` and see what it's
complaining about. This will give you a hint on what you need to do to fix
your environment.

