title: Scala Spark Streaming job with spark-submit gets RECEIVED SIGNAL TERM error
body: <p>I am running a spark streaming job using <code>spark-submit</code> on an EMR node in AWS. This streams from kafka topic and outputs to another kafka topic. The job is written in <strong>scala.</strong></p>
  
  <p>I execute the job like this:</p>
  
  <pre><code>spark-submit \
  --master yarn \
  --num-executors 2 \
  --total-executor-cores 1 \
  --executor-cores 1 \
  --executor-memory 1g \
  --driver-memory 1g \
  --name my-spark-job \
  --class com.example.Main \
  --queue default \
  --conf "spark.dynamicAllocation.enabled=true" \
  --conf "spark.shuffle.service.enabled=true" \
  --conf "spark.driver.memoryOverhead=1024m" \
  --conf "spark.executor.memoryOverhead=1024m" \
  --deploy-mode cluster 
  spark-job-assembly-0.0.1-SNAPSHOT.jar
  </code></pre>
  
  <p>I see in logs that the job is sent to yarn and it gets this <strong>ERROR SIGNAL TERM</strong>:</p>
  
  <pre><code>20/05/06 10:41:04 INFO spark.ContextCleaner: Cleaned accumulator 181
  20/05/06 10:41:09 ERROR yarn.ApplicationMaster: RECEIVED SIGNAL TERM
  20/05/06 10:41:09 INFO streaming.StreamingContext: Invoking stop(stopGracefully=false) from shutdown hook
  20/05/06 10:41:09 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
  20/05/06 10:41:09 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
  20/05/06 10:41:09 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1588761660000
  20/05/06 10:41:09 INFO scheduler.JobGenerator: Stopped JobGenerator
  20/05/06 10:41:09 INFO scheduler.JobScheduler: Stopped JobScheduler
  20/05/06 10:41:09 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@3fa68e57{/streaming,null,UNAVAILABLE,@Spark}
  20/05/06 10:41:09 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@446023a8{/streaming/batch,null,UNAVAILABLE,@Spark}
  20/05/06 10:41:09 INFO handler.ContextHandler: Stopped o.s.j.s.ServletContextHandler@4e16f11a{/static/streaming,null,UNAVAILABLE,@Spark}
  20/05/06 10:41:09 INFO streaming.StreamingContext: StreamingContext stopped successfully
  20/05/06 10:41:09 INFO spark.SparkContext: Invoking stop() from shutdown hook
  20/05/06 10:41:09 INFO yarn.ApplicationMaster: Final app status: SUCCEEDED, exitCode: 0
  20/05/06 10:41:09 INFO server.AbstractConnector: Stopped Spark@5bec9336{HTTP/1.1,[http/1.1]}{0.0.0.0:0}
  
  .....
  ... further down i get ...
  ...
  20/05/06 10:41:00 INFO storage.BlockManager: Removing RDD 7
  20/05/06 10:41:00 INFO storage.BlockManager: Removing RDD 6
  20/05/06 10:41:09 ERROR executor.CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
  20/05/06 10:41:09 INFO storage.DiskBlockManager: Shutdown hook called
  20/05/06 10:41:09 INFO util.ShutdownHookManager: Shutdown hook called
  </code></pre>
  
  <p>Is there something I need to set in the config or anything else I may be missing? Would appreciate any hints here, thanks!</p>