title: Calculating month differences in pyspark between timestamp and a string
body: <p>I am using pyspark in databricks with a JSON file to clean data. The expression in the <code>eval</code> brackets comes from the JSON file.</p>
  
  <p>One of the issues I am facing is manipulating <code>timestamp</code>s/<code>string</code>.</p>
  
  <p>I am trying to find the difference in months between a timestamp column and a single date (which is a string)</p>
  
  <p>See code below. </p>
  
  <pre><code>import pyspark.sql.functions as F
  df2 = df2.withColumn('test', eval("months_between( F.to_date(F.col('period_name')), lit('31/03/2019'))"))
  </code></pre>
  
  <p>It doesn't throw an error but evaluates to null.</p>