title: Use Spring together with Spark
body: <p>I'm developing a Spark Application and I'm used to Spring as a Dependency Injection Framework. Now I'm stuck with the problem, that the processing part uses the @Autowired functionality of Spring, but it is serialized and deserialized by Spark.</p>
  
  <p>So the following code gets me into trouble:</p>
  
  <pre><code>Processor processor = ...; // This is a Spring constructed object
                             // and makes all the trouble
  JavaRDD&lt;Txn&gt; rdd = ...; // some data for Spark
  rdd.foreachPartition(processor);
  </code></pre>
  
  <p>The Processor looks like that:</p>
  
  <pre><code>public class Processor implements VoidFunction&lt;Iterator&lt;Txn&gt;&gt;, Serializeable {
      private static final long serialVersionUID = 1L;
  
      @Autowired // This will not work if the object is deserialized
      private transient DatabaseConnection db;
  
      @Override
      public void call(Iterator&lt;Txn&gt; txns) {
          ... // do some fance stuff
          db.store(txns);
      }
  }
  </code></pre>
  
  <p>So my question is: Is it even possible to use something like Spring in combination with Spark? If not, what is the most elegant way to do something like that? Any help is appreciated!</p>