title: Parsing XML column in Pyspark dataframe
body: <p>I'm relatively new to PySpark and trying to solve a data problem. I have a pyspark DF, created with data extracted from MS SQL Server, having 2 columns: ID (Integer) and XMLMsg (String). The 2nd column, XMLMsg contains data in XML format.
  The goal is to parse the XMLMsg column and create additional columns in the same DF with the extracted columns from the XML.</p>
  <p>Following is a sample structure of the pyspark DF:</p>
  <pre><code>ID  XMLMsg
  101 ...&lt;a&gt;&lt;b&gt;name1&lt;/b&gt;&lt;c&gt;loc1&lt;/c&gt;&lt;/a&gt;...&lt;d&gt;dept1&lt;/d&gt;...
  102 ...&lt;a&gt;&lt;b&gt;name2&lt;/b&gt;&lt;c&gt;loc2&lt;/c&gt;&lt;/a&gt;...&lt;d&gt;dept2&lt;/d&gt;...
  103 ...&lt;a&gt;&lt;b&gt;name3&lt;/b&gt;&lt;c&gt;loc3&lt;/c&gt;&lt;/a&gt;...&lt;d&gt;dept3&lt;/d&gt;...
  </code></pre>
  <p>Expected output is:</p>
  <pre><code>ID  XMLMsg                                              b       c       d
  101 ...&lt;a&gt;&lt;b&gt;name1&lt;/b&gt;&lt;c&gt;loc1&lt;/c&gt;&lt;/a&gt;...&lt;d&gt;dept1&lt;/d&gt;... name1   loc1    dept1
  102 ...&lt;a&gt;&lt;b&gt;name2&lt;/b&gt;&lt;c&gt;loc2&lt;/c&gt;&lt;/a&gt;...&lt;d&gt;dept2&lt;/d&gt;... name2   loc2    dept2
  103 ...&lt;a&gt;&lt;b&gt;name3&lt;/b&gt;&lt;c&gt;loc3&lt;/c&gt;&lt;/a&gt;...&lt;d&gt;dept3&lt;/d&gt;... name3   loc3    dept3
  </code></pre>
  <p>I tried few suggestions based on my search in SO; however, could not achieve the expected result. Hence, reaching out for some help and directions. Thanks for your time.</p>