title: How to parse nested xml in pyspark
body: <p>I have the following xml document</p>
  <pre><code>&lt;a date=&quot;26-03-2018&quot; id=&quot;1&quot;&gt;
  &lt;text&gt;
  &lt;/text&gt;
  &lt;metadata&gt;
  &lt;b&gt;
  &lt;c c=&quot;STRING1&quot;&gt;
  &lt;d=&quot;value&quot; e=&quot;string&quot;/&gt;
  &lt;/c&gt;
  &lt;c c=&quot;STRING2&quot;&gt;
  &lt;d=&quot;value2&quot; e=&quot;string&quot; /&gt;
  &lt;/c&gt;
  &lt;/b&gt;
  &lt;/metadata&gt;
  &lt;/a&gt;
  </code></pre>
  <p>By using data bricks xml parser,I want to extract the <code>string1</code>,<code>string2</code> values of <code>c</code> as a list to the <code>column[metadata]</code> of dataframe but when I infer with custom schema</p>
  <pre><code>schema = StructType([
  StructField(&quot;date&quot;, StringType(), True),
  StructField(&quot;id&quot;, LongType(), True),
  StructField(&quot;text&quot;, StringType(), True),
  StructField(&quot;metadata&quot;, StructType([
  StructField(&quot;b&quot;, StringType(), True)]), True),])
  </code></pre>
  <p>and the dataframe for the above schema</p>
  <pre><code>----------------------------------------------------------------------------------------------------------------------
   Id | date       | text | metadata 
  ----------------------------------------------------------------------------------------------------------------------
   1  | 26-03-2018 | text |' &lt;c c=&quot;STRING1&quot;&gt; &lt;d=&quot;value&quot; e=&quot;string&quot;/&gt;&lt;/c&gt;&lt;c c=&quot;STRING2&quot;&gt;&lt;d=&quot;value2&quot; e=&quot;string&quot; /&gt; &lt;/c&gt;'
  </code></pre>
  <p>I am getting the entire data as string from <code>b</code> node. Any ideas on how to extract only strings using databricks xml parser to the column named metadata or is there any other parser available. I couldn't find the correct solution.</p>