title: Apache Flink DataSet difference/subtraction operation
body: <p>I'm currently trying to implement some algorithms in both, Apache Spark and Apache Flink. When executing the algorithms, I have to do some kind of set difference/subtraction operations.</p>
  
  <p>While there is a built-in <code>subtract</code> operation for Apache Spark, I couldn't find something similar in Apache Flink (1.0.3 and 1.1.0-SNAPSHOT).</p>
  
  <p>So my question is, given two DataSet objects <code>d1, d2</code> both containing the same type <code>T</code>, what is the most efficient way to apply set difference, i.e. <code>d1\d2</code>?</p>
  
  <pre><code>val d1: DataSet[T] = ...
  val d2: DataSet[T] = ...
  val d_diff: DataSet[T] = ???
  </code></pre>
  
  <p>Probably there is some way to it via <code>coGroup</code></p>
  
  <pre><code>val d_diff = d1.coGroup(d2).where(0).equalTo(0) {
                 (l, r, out: Collector[T]) =&gt; {
                  val rightElements = r.toSet
                  for (el &lt;- l)
                    if (!rightElements.contains(el)) out.collect(el)
                 }
               }
  </code></pre>
  
  <p>but I'm wondering whether that's the correct way or even best-practice or does anybody know some more efficient way?</p>