title: Spark Streaming -> DStream.checkpoint versus SparkStreaming.checkpoint
body: <p>I have Spark 1.4 Streaming application, which reads data from Kafka, uses statefull transformation, and has batch interval of 15 seconds.</p>
  
  <p>In order to use statefull transformations, as well as recover from driver failures, I need to set checkpointing on streaming context.</p>
  
  <p>Also, in the Spark 1.4 documentation they recommend DStream checkpointing 5-10 times of the batch interval.</p>
  
  <p>So my questions is:</p>
  
  <p>What happens if I only set checkpointing on spark streaming context? I guess DStreams will be checkpointed every batch interval?</p>
  
  <p>What If I set both checkpointing on streaming context as well as the moment I read data from Kafka, I set:</p>
  
  <blockquote>
    <p>DStream.checkpoint(90 seconds)</p>
  </blockquote>
  
  <p>What will be the intervals for metadata checkpointing and what for data checkpointing (meaning DStreams)? </p>
  
  <p>Thank you.</p>