title: SELECT DISTINCT Cassandra in Spark
body: <p>I need a query that lists out the the unique <em>Composite Partition Keys</em> inside of spark.<br>
  The query in CASSANDRA: <code>SELECT DISTINCT key1, key2, key3 FROM schema.table;</code> is quite fast, however putting the same sort of data filter in a RDD or spark.sql retrieves results incredibly slowly in comparison.</p>
  
  <p>e.g.  </p>
  
  <pre><code>---- SPARK ----
  var t1 = sc.cassandraTable("schema","table").select("key1", "key2", "key3").distinct()
  var t2 = spark.sql("SELECT DISTINCT key1, key2, key3 FROM schema.table")
  
  t1.count // takes 20 minutes
  t2.count // takes 20 minutes
  
  ---- CASSANDRA ----
  // takes &lt; 1 minute while also printing out all results
  SELECT DISTINCT key1, key2, key3 FROM schema.table; 
  </code></pre>
  
  <p>where the table format is like:</p>
  
  <pre><code>CREATE TABLE schema.table (
      key1 text,
      key2 text,
      key3 text,
      ckey1 text,
      ckey2 text,
      v1 int,
      PRIMARY KEY ((key1, key2, key3), ckey1, ckey2)
  );
  </code></pre>
  
  <p>Doesn't spark use cassandra optimisations in its' queries?<br>
  How can I retreive this information efficiently?</p>