title: Subtract values of columns from two different data frames in PySpark to find RMSE
body: <p>I am not able to figure it out. I am trying to calculate the RMSE between test and prediction data.</p>
  
  <p>test   </p>
  
  <pre><code>col1    col2
   a        2 
   b        3
  </code></pre>
  
  <p>prediction</p>
  
  <pre><code>col1   col2
   a       4 
   b       5
  </code></pre>
  
  <p>I am trying to do this test(col2)-prediction(col2). That is </p>
  
  <pre><code>2-4 =-2
  3-5 =-2
  </code></pre>
  
  <p>I tried </p>
  
  <pre><code>test.select("col2").subtract(prediction.select("col2"))
  </code></pre>
  
  <p>But I am not getting the required result. I am trying to obtain this result to find the RMSE. Is there a built in function in spark to find the RMSE?</p>
  
  <p>Thank you.</p>