title: pySpark/Python iterate through dataframe columns, check for a condition and populate another colum
body: <p>I am working with python/pySpark in Jupyter Notebook and I am trying to figure out the following:</p>
  
  <p>I've got a dataframe like</p>
  
  <pre><code>MainDate      Date1        Date2        Date3         Date4
  2015-10-25    2015-09-25   2015-10-25   2015-11-25    2015-12-25
  2012-07-16    2012-04-16   2012-05-16   2012-06-16    2012-07-16
  2005-03-14    2005-07-14   2005-08-14   2005-09-14    2005-10-14
  </code></pre>
  
  <p>I need to compare MainDate with each of the Date1-Date4 and if MainDate == Date# then to create a new column REAL = Date#, if there is no match then REAL = "None", all the dates are in Date format, also the real dataframe has Date1 to Date72 and there could be only one match, if there is any</p>
  
  <p>Final result:</p>
  
  <pre><code>MainDate      Date1        Date2        Date3         Date4        REAL
  2015-10-25    2015-09-25   2015-10-25   2015-11-25    2015-12-25   Date2
  2012-07-16    2012-04-16   2012-05-16   2012-06-16    2012-07-16   Date4
  2005-03-14    2005-07-14   2005-08-14   2005-09-14    2005-10-14   None
  </code></pre>
  
  <p>Thanks in advance</p>