title: Spark streaming + Kafka vs Just Kafka
body: <p>Why and when one would choose to use Spark streaming with Kafka?</p>
  
  <p>Suppose I have a system getting thousand messages per seconds through Kafka. I need to apply some real time analytics on these messages and store the result in a DB.</p>
  
  <p>I have two options:</p>
  
  <ol>
  <li><p>Create my own worker that reads messages from Kafka, run the analytics algorithm and store the result in DB. In a Docker era it is easy to scale this worker through my entire cluster with just scale command. I just need to make sure I have an equal or grater number of partitions than my workers and all is good and I have a true concurrency. </p></li>
  <li><p>Create a Spark cluster with Kafka streaming input. Let the Spark cluster to do the analytics computations and then store the result.</p></li>
  </ol>
  
  <p>Is there any case when the second option is a better choice? Sounds to me like it is just an extra overhead.</p>