title: How to join two dataframes?
body: <p>I cannot get Sparks DataFrame join to work (no result gets produced). Here is my code:</p>
  
  <pre><code>val e = Seq((1, 2), (1, 3), (2, 4))
  var edges = e.map(p =&gt; Edge(p._1, p._2)).toDF()
  var filtered = edges.filter("start = 1").distinct()
  println("filtered")
  filtered.show()
  filtered.printSchema()
  println("edges")
  edges.show()
  edges.printSchema()
  var joined = filtered.join(edges, filtered("end") === edges("start"))//.select(filtered("start"), edges("end"))
  println("joined")
  joined.show()
  </code></pre>
  
  <p>It requires <code>case class Edge(start: Int, end: Int)</code> to be defined at top level. Here is the output it produces:</p>
  
  <pre><code>filtered
  +-----+---+
  |start|end|
  +-----+---+
  |    1|  2|
  |    1|  3|
  +-----+---+
  
  root
   |-- start: integer (nullable = false)
   |-- end: integer (nullable = false)
  
  edges
  +-----+---+
  |start|end|
  +-----+---+
  |    1|  2|
  |    1|  3|
  |    2|  4|
  +-----+---+
  
  root
   |-- start: integer (nullable = false)
   |-- end: integer (nullable = false)
  
  joined
  +-----+---+-----+---+
  |start|end|start|end|
  +-----+---+-----+---+
  +-----+---+-----+---+
  </code></pre>
  
  <p>I don't understand why the output is empty. Why isn't the first row of <code>filtered</code> get combined with the last row of <code>edges</code>?</p>