title: Convert a JSON string to a struct column without schema in Spark
body: <p>Spark: 3.0.0<br />
  Scala: 2.12.8</p>
  <p>My data frame has a column with JSON string, and I want to create a new column from it with the StructType.</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>temp_json_string</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>{&quot;name&quot;:&quot;test&quot;,&quot;id&quot;:&quot;12&quot;,&quot;category&quot;:[{&quot;products&quot;:[&quot;A&quot;,&quot;B&quot;],&quot;displayName&quot;:&quot;test_1&quot;,&quot;displayLabel&quot;:&quot;test1&quot;},{&quot;products&quot;:[&quot;C&quot;],&quot;displayName&quot;:&quot;test_2&quot;,&quot;displayLabel&quot;:&quot;test2&quot;}],&quot;createdAt&quot;:&quot;&quot;,&quot;createdBy&quot;:&quot;&quot;}</td>
  </tr>
  </tbody>
  </table>
  </div>
  <pre class="lang-none prettyprint-override"><code>root
   |-- temp_json_string: string (nullable = true)
  </code></pre>
  <p>Formatted JSON:</p>
  <pre class="lang-json prettyprint-override"><code>{
    &quot;name&quot;:&quot;test&quot;,
    &quot;id&quot;:&quot;12&quot;,
    &quot;category&quot;:[
      {
        &quot;products&quot;:[
          &quot;A&quot;,
          &quot;B&quot;
        ],
        &quot;displayName&quot;:&quot;test_1&quot;,
        &quot;displayLabel&quot;:&quot;test1&quot;
      },
      {
        &quot;products&quot;:[
          &quot;C&quot;
        ],
        &quot;displayName&quot;:&quot;test_2&quot;,
        &quot;displayLabel&quot;:&quot;test2&quot;
      }
    ],
    &quot;createdAt&quot;:&quot;&quot;,
    &quot;createdBy&quot;:&quot;&quot;
  }
  </code></pre>
  <p>I want to create a new column of type Struct so I tried:</p>
  <pre><code>dataFrame
       .withColumn(&quot;temp_json_struct&quot;, struct(col(&quot;temp_json_string&quot;)))
       .select(&quot;temp_json_struct&quot;)
  </code></pre>
  <p>Now, I get the schema as:</p>
  <pre class="lang-none prettyprint-override"><code>root
   |-- temp_json_struct: struct (nullable = false)
   |    |-- temp_json_string: string (nullable = true)
  </code></pre>
  <p>Desired result:</p>
  <pre class="lang-none prettyprint-override"><code>root
   |-- temp_json_struct: struct (nullable = false)
   |    |-- name: string (nullable = true)
   |    |-- category: array (nullable = true)
   |    |    |-- products: array (nullable = true)
   |    |    |-- displayName: string (nullable = true)
   |    |    |-- displayLabel: string (nullable = true)
   |    |-- createdAt: timestamp (nullable = true)
   |    |-- updatedAt: timestamp (nullable = true)
  </code></pre>