title: Filter out rows with NaN values for certain column
body: <p>I have a dataset and in some of the rows an attribute value is <code>NaN</code>. This data is loaded into a dataframe and I would like to only use the rows which consist of rows where all attribute have values. I tried doing it via sql:</p>
  
  <pre><code>val df_data = sqlContext.sql("SELECT * FROM raw_data WHERE attribute1 != NaN")
  </code></pre>
  
  <p>I tried several variants on this, but I can't seem to get it working. </p>
  
  <p>Another option would be to transform it to a RDD and then filter it, since filtering this dataframe to check if a attribute <code>isNaN</code> , does not work.</p>