title: Spark Streaming with Python - class not found exception
body: <p>I'm working on a project to bulk load data from a CSV file to HBase using Spark streaming. The code I'm using is as follows (adapted from <a href="https://stackoverflow.com/questions/32175634/pyspark-can-saveasnewapihadoopdataset-be-used-as-bulk-loading-to-hbase">here</a>):</p>
  
  <pre><code>def bulk_load(rdd):
      conf = {#removed for brevity}
  
      keyConv = "org.apache.spark.examples.pythonconverters.StringToImmutableBytesWritableConverter"
      valueConv = "org.apache.spark.examples.pythonconverters.StringListToPutConverter"
  
      load_rdd = rdd.flatMap(lambda line: line.split("\n"))\
                    .flatMap(csv_to_key_value)
      load_rdd.saveAsNewAPIHadoopDataset(conf=conf,keyConverter=keyConv,valueConverter=valueConv)
  </code></pre>
  
  <p>Everything up to and including the two <code>flatMap</code>s works as expected. However, when trying to execute <code>saveAsNewAPIHadoopDataset</code> I get the following runtime error:</p>
  
  <p><code>java.lang.ClassNotFoundException: org.apache.spark.examples.pythonconverters.StringToImmutableBytesWritableConverter</code></p>
  
  <p>I have set <code>PYTHONPATH</code> to point to the jar containing this class (as well as my other converter class) however this does not seem to have improved the situation. Any advice would be greatly appreciated. Thanks in advance.</p>