title: Sudden Increase in Scheduling delay for spark streaming Job Without change in other parameters
body: <p>I have a spark streaming job running in production with 1 sec batches. I use CDH 5.5 Spark 1.5.  We use Kafka Create Directstream. We have enabled Back Pressure. We dont want to sue dynamic allocation So the job executed with fix number of executor.</p>
  
  <p>From the <a href="https://i.stack.imgur.com/onbif.jpg" rel="nofollow noreferrer">below image</a> i can see that these is sudden increase in the scheduling delay from 13.50 But during the same time i dont see any dealy in the processing time.</p>
  
  <ol>
  <li>What would be possible reasons for increase in the scheduling time whne processing times are same.</li>
  <li>Does other job loads in the cluster effect the current streaming job. In my understanding it should not be the case because the executors for streaming are pre-allocated and are already running </li>
  </ol>
  
  <p>Any thoughts?</p>
  
  <p><a href="https://i.stack.imgur.com/onbif.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/onbif.jpg" alt="enter image description here"></a></p>