title: pyspark: how to partition by date column in format 'yyyy-MM-dd HH'
body: <p>I have tried the following:</p>
  <pre><code>df = (spark.createDataFrame([(1, '2020-12-03 01:01:01'), (2, '2022-11-04 10:10:10'),], ['id', 'txt'])
    .withColumn(&quot;testCol&quot;, to_timestamp(col(&quot;txt&quot;), &quot;yyyy-MM-dd HH&quot;)))
  </code></pre>
  <p><a href="https://i.stack.imgur.com/4P2dN.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/4P2dN.png" alt="result" /></a></p>
  <p>I basically want a timestamp/datetime column in the format (yyyy-MM-dd HH). The above piece of code gives the following result as shown in image. But when I try to write this to Azure Blob Storage partitioned by this time column then it gives some garbage like:</p>
  <p><a href="https://i.stack.imgur.com/KWl9h.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/KWl9h.png" alt="final result" /></a></p>
  <p>Is there any cleaner way to do this such that the column format remains timestamp/datetime in the format (yyyy-MM-dd HH) and at same time while writing the partition it looks clean in the same way and not the garbage strings of '%3A55%....'</p>
  <p>Thanks.</p>