title: pyspark: Converting string to struct
body: <p>I have data as follows -</p>
  
  <pre><code>{
      "Id": "01d3050e",
      "Properties": "{\"choices\":null,\"object\":\"demo\",\"database\":\"pg\",\"timestamp\":\"1581534117303\"}",
      "LastUpdated": 1581530000000,
      "LastUpdatedBy": "System"
  }
  </code></pre>
  
  <p>Using aws glue, I want to relationalize the "Properties" column  but since the datatype is string it can't be done. Converting it to struct, might do it based on reading this blog -</p>
  
  <p><a href="https://aws.amazon.com/blogs/big-data/simplify-querying-nested-json-with-the-aws-glue-relationalize-transform/" rel="nofollow noreferrer">https://aws.amazon.com/blogs/big-data/simplify-querying-nested-json-with-the-aws-glue-relationalize-transform/</a></p>
  
  <pre><code>&gt;&gt;&gt; df.show
  &lt;bound method DataFrame.show of DataFrame[Id: string, LastUpdated: bigint, LastUpdatedBy: string, Properties: string]&gt;
  &gt;&gt;&gt; df.show()
  +--------+-------------+-------------+--------------------+
  |      Id|  LastUpdated|LastUpdatedBy|          Properties|
  +--------+-------------+-------------+--------------------+
  |01d3050e|1581530000000|       System|{"choices":null,"...|
  +--------+-------------+-------------+--------------------+
  
  </code></pre>
  
  <p>How can I un-nested the "properties" column to break it into "choices", "object", "database" and "timestamp" columns, using relationalize transformer or any UDF in pyspark.</p>