title: Spark Streaming: Using PairRDD.saveAsNewHadoopDataset function to save data to HBase
body: <p>I want to save a Twitter stream in a HBase database. What I have now, is the Saprk Application to receive and transform the data. But I don't know how to save my TwitterStream into HBase?</p>
  
  <p>The only thing I found that could be useful is the <code>PairRDD.saveAsNewAPIHadoopDataset(conf)</code> method. But how shall I use it, which Configurations do I have to make to able to save the RDD data to my HBase table?</p>
  
  <p>The only thing I found yet is the HBase client library, which can insert data to a table via Put objects. But this isn't a solution for inside a Spark program, is it (would be necessary to iterate over all items inside the RDD!!)? </p>
  
  <p>Can someone give an example in JAVA? My main problem seems to be the set-up of the <code>org.apache.hadoop.conf.Configuration</code> instance, I have to submit in the <code>saveAsNewAPIHadoopDataset</code>...</p>
  
  <p>Here a code snippet:</p>
  
  <pre><code>JavaReceiverInputDStream&lt;Status&gt; statusDStream = TwitterUtils.createStream(streamingCtx);
  
  JavaPairDStream&lt;Long, String&gt; statusPairDStream = statusDStream.mapToPair(new PairFunction&lt;Status, Long, String&gt;() {
      public Tuple2&lt;Long, String&gt; call(Status status) throws Exception {
          return new Tuple2&lt;Long, String&gt; (status.getId(), status.getText());
      }
  });
  
  statusPairDStream.foreachRDD(new Function&lt;JavaPairRDD&lt;Long,String&gt;, Void&gt;() {
      public Void call(JavaPairRDD&lt;Long, String&gt; status) throws Exception {
          org.apache.hadoop.conf.Configuration conf = new Configuration();
          status.saveAsNewAPIHadoopDataset(conf);
          // HBase PUT here can't be correct!?
          return null;
      }
  });
  </code></pre>