title: Spark worker shutdown - how to free shared resources
body: <p>In Spark manual there is recommended to use <em>shared static resource</em> (e. g. connection pool) inside the worker code.</p>
  
  <p>Example from the <a href="http://spark.apache.org/docs/latest/streaming-programming-guide.html#design-patterns-for-using-foreachrdd" rel="noreferrer">manual</a>:</p>
  
  <pre class="lang-scala prettyprint-override"><code>dstream.foreachRDD { rdd =&gt;
    rdd.foreachPartition { partitionOfRecords =&gt;
      // ConnectionPool is a static, lazily initialized pool of connections
      val connection = ConnectionPool.getConnection()
      partitionOfRecords.foreach(record =&gt; connection.send(record))
      ConnectionPool.returnConnection(connection)  // return to the pool for future reuse
    }
  }
  </code></pre>
  
  <p>What to do when the <strong>static resource</strong> needs to be freed/closed before the executor is shut down? There is no place where to call the <code>close()</code> function. Tried a shutdown hook, but it doesn't seem to help.</p>
  
  <p>Actually currently my worker process becomes <em>zombie</em>, because I am using shared resource which creates a pool of non-deamon threads (HBase async client) that means the JVM hangs until forever.</p>
  
  <p>I am using the Spark Streaming <em>graceful shutdown</em> called on the driver:</p>
  
  <pre class="lang-java prettyprint-override"><code>streamingContext.stop(true, true);
  </code></pre>
  
  <hr>
  
  <p>EDIT:</p>
  
  <p>It seems there already exists an issue in the Spark JIRA that's dealing with the same problem</p>
  
  <p><a href="https://issues.apache.org/jira/browse/SPARK-10911" rel="noreferrer">https://issues.apache.org/jira/browse/SPARK-10911</a></p>