title: Filtering on multiple columns in Spark dataframes
body: <p>Suppose I have a dataframe in Spark as shown below -</p>
  
  <pre><code>val df = Seq(
  (0,0,0,0.0),
  (1,0,0,0.1),
  (0,1,0,0.11),
  (0,0,1,0.12),
  (1,1,0,0.24),
  (1,0,1,0.27),
  (0,1,1,0.30),
  (1,1,1,0.40)
  ).toDF("A","B","C","rate")
  </code></pre>
  
  <p>Here is how it looks like - </p>
  
  <pre><code>scala&gt; df.show()
  +---+---+---+----+
  |  A|  B|  C|rate|
  +---+---+---+----+
  |  0|  0|  0| 0.0|
  |  1|  0|  0| 0.1|
  |  0|  1|  0|0.11|
  |  0|  0|  1|0.12|
  |  1|  1|  0|0.24|
  |  1|  0|  1|0.27|
  |  0|  1|  1| 0.3|
  |  1|  1|  1| 0.4|
  +---+---+---+----+
  </code></pre>
  
  <p>A,B and C are the advertising channels in this case. 0 and 1 represent absence and presence of channels respectively. 2^3 shows 8 combinations in the data-frame.</p>
  
  <p>I want to filter records from this data-frame that shows presence of 2 channels at a time( AB, AC, BC) . Here is how I want my output to be -</p>
  
  <pre><code>+---+---+---+----+
  |  A|  B|  C|rate|
  +---+---+---+----+
  |  1|  1|  0|0.24|
  |  1|  0|  1|0.27|
  |  0|  1|  1| 0.3|
  +---+---+---+----+
  </code></pre>
  
  <p>I can write 3 statements to get the output by doing - </p>
  
  <pre><code>scala&gt; df.filter($"A" === 1 &amp;&amp; $"B" === 1 &amp;&amp; $"C" === 0).show()
  +---+---+---+----+
  |  A|  B|  C|rate|
  +---+---+---+----+
  |  1|  1|  0|0.24|
  +---+---+---+----+
  
  
  scala&gt; df.filter($"A" === 1 &amp;&amp; $"B" === 0  &amp;&amp; $"C" === 1).show()
  +---+---+---+----+
  |  A|  B|  C|rate|
  +---+---+---+----+
  |  1|  0|  1|0.27|
  +---+---+---+----+
  
  
  scala&gt; df.filter($"A" === 0 &amp;&amp; $"B" === 1 &amp;&amp; $"C" === 1).show()
  +---+---+---+----+
  |  A|  B|  C|rate|
  +---+---+---+----+
  |  0|  1|  1| 0.3|
  +---+---+---+----+
  </code></pre>
  
  <p>However, I want to achieve this using either a single statement that does my job or a function that helps me get the output.
  I was thinking of using a case statement to match the values. However in general my dataframe might consist of more than 3 channels - </p>
  
  <pre><code>scala&gt; df.show()
  +---+---+---+---+----+
  |  A|  B|  C|  D|rate|
  +---+---+---+---+----+
  |  0|  0|  0|  0| 0.0|
  |  0|  0|  0|  1| 0.1|
  |  0|  0|  1|  0| 0.1|
  |  0|  0|  1|  1|0.59|
  |  0|  1|  0|  0| 0.1|
  |  0|  1|  0|  1|0.89|
  |  0|  1|  1|  0|0.39|
  |  0|  1|  1|  1| 0.4|
  |  1|  0|  0|  0| 0.0|
  |  1|  0|  0|  1|0.99|
  |  1|  0|  1|  0|0.49|
  |  1|  0|  1|  1| 0.1|
  |  1|  1|  0|  0|0.79|
  |  1|  1|  0|  1| 0.1|
  |  1|  1|  1|  0| 0.1|
  |  1|  1|  1|  1| 0.1|
  +---+---+---+---+----+
  </code></pre>
  
  <p>In this scenario I would want my output as -</p>
  
  <pre><code>scala&gt; df.show()
  +---+---+---+---+----+
  |  A|  B|  C|  D|rate|
  +---+---+---+---+----+
  |  0|  0|  1|  1|0.59|
  |  0|  1|  0|  1|0.89|
  |  0|  1|  1|  0|0.39|
  |  1|  0|  0|  1|0.99|
  |  1|  0|  1|  0|0.49|
  |  1|  1|  0|  0|0.79|
  +---+---+---+---+----+
  </code></pre>
  
  <p>which shows rates for paired presence of channels => (AB, AC, AD, BC, BD, CD).</p>
  
  <p>Kindly help.</p>