title: spark cassandra intelliJ scala
body: <p>Environment:</p>
  
  <p>My Laptop:</p>
  
  <pre><code>IntelliJ IDEA 2017.1.4
  JRE: 1.8.0_112-release-736-b21 x86_64
  JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
  Mac OS X 10.12.2
  IP Address: 192.168.3.203
  </code></pre>
  
  <p>AWS Environment:</p>
  
  <pre><code>Linux ip-172-31-16-45  x86_64 x86_64 GNU/Linux
  DSE Version : 5.1.0
  IP Address: 172.31.16.45
  PUBLIC IP: 35.35.42.42 
  </code></pre>
  
  <p>Ecexuted the following commands:</p>
  
  <pre><code>[ec2-user@ip-172-31-16-45 ~]$ dse cassandra -k 
  and then
  [ec2-user@ip-172-31-16-45 ~]$ dse spark
  
  The log file is at /home/ec2-user/.spark-shell.log
  warning: there was one deprecation warning; re-run with -deprecation for details
  New Spark Session
  WARN  2017-09-13 13:18:26,907 org.apache.spark.SparkContext: Use an existing SparkContext, some configuration may not take effect.
  Extracting Spark Context
  Extracting SqlContext
  Spark context Web UI available at http://172.31.16.45:4040
  Spark context available as 'sc' (master = dse://?, app id = app-20170913131826-0001).
  Spark session available as 'spark'.
  
  Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112)
  Type in expressions to have them evaluated.
  Type :help for more information.
  
  scala&gt;
  </code></pre>
  
  <p>Spark Version: 2.0.2.6 ( One I see on Web Browser when I open <a href="http://35.35.42.42:4040/executors/" rel="nofollow noreferrer">http://35.35.42.42:4040/executors/</a>)
  Both Spark and Cassandra database are running on 35.35.42.42(public IP) or (Local IP: 172.31.16.45)
  When I continue coding from scala env on AWS my code is running fine. What I need is to run the same code from intelliJ . Is it possible. If not what is the process.</p>
  
  <p>Now I opened IntelliJ on my laptop and tried running below code:</p>
  
  <p>// In  .setMaster I tried all - .setMaster ("all combinations of local IP, public IP, spark, dse etc")</p>
  
  <pre><code>import org.apache.spark.SparkContext
  import org.apache.spark.SparkContext._
  import org.apache.spark.SparkConf
  import com.datastax.spark.connector._
  
  object helloworld {
  
    def main(args: Array[String]) {
      val conf = new SparkConf().setAppName("Simple Application").setMaster("dse://35.35.42.42:4040")
  //    val conf = new SparkConf().setAppName("Simple Application").setMaster("spark://172.31.16.45:4040")
  //    val conf = new SparkConf().setAppName("Simple Application").setMaster("spark://35.35.42.42:7077")
      val conf = new SparkConf().setAppName("Simple Application").setMaster("spark://172.31.16.45:7077")
  
      val sc = new SparkContext(conf)
  //    val rdd = sc.cassandraTable("tdata", "map")
  
    }
  }
  </code></pre>
  
  <p>Error:
  org.apache.spark.SparkException: Could not parse Master URL: 'dse://35.35.42.42:4040'</p>