title: Count the number of non-null values in a Spark DataFrame
body: <p>I have a data frame with some columns, and before doing analysis, I'd like to understand how complete such data frame is, so I want to filter the data frame and count for each column the number of non-null values, possibly returning a dataframe back.</p>
  
  <p>Basically, I am trying to achieve the same result as expressed in <a href="https://stackoverflow.com/questions/29971075/count-number-of-non-nan-entries-in-every-column-of-dataframe">this question</a> but using Scala instead of Python... </p>
  
  <p>Say you have:</p>
  
  <pre class="lang-scala prettyprint-override"><code>val row = Row("x", "y", "z")
  val df = sc.parallelize(Seq(row(0, 4, 3), row(None, 3, 4), row(None, None, 5))).toDF()
  </code></pre>
  
  <p>How can you summarize the number of non-null for each column and return a dataframe with the same number of column and just a single row with the answer?</p>