title: Parsing through a deep-nested XML File in Python
body: <p>I am looking at an xml file similar to the below:</p>
  
  <pre><code>&lt;pinnacle_line_feed&gt;
    &lt;PinnacleFeedTime&gt;1418929691920&lt;/PinnacleFeedTime&gt;
    &lt;lastContest&gt;28962804&lt;/lastContest&gt;
    &lt;lastGame&gt;162995589&lt;/lastGame&gt;
    &lt;events&gt;
      &lt;event&gt;
        &lt;event_datetimeGMT&gt;2014-12-19 11:15&lt;/event_datetimeGMT&gt;
        &lt;gamenumber&gt;422739932&lt;/gamenumber&gt;
        &lt;sporttype&gt;Alpine Skiing&lt;/sporttype&gt;
        &lt;league&gt;DH 145&lt;/league&gt;
        &lt;IsLive&gt;No&lt;/IsLive&gt;
        &lt;participants&gt;
          &lt;participant&gt;
            &lt;participant_name&gt;Kjetil Jansrud (NOR)&lt;/participant_name&gt;
            &lt;contestantnum&gt;2001&lt;/contestantnum&gt;
            &lt;rotnum&gt;2001&lt;/rotnum&gt;
            &lt;visiting_home_draw&gt;Visiting&lt;/visiting_home_draw&gt;
          &lt;/participant&gt;
          &lt;participant&gt;
            &lt;participant_name&gt;The Field&lt;/participant_name&gt;
            &lt;contestantnum&gt;2002&lt;/contestantnum&gt;
            &lt;rotnum&gt;2002&lt;/rotnum&gt;
            &lt;visiting_home_draw&gt;Home&lt;/visiting_home_draw&gt;
          &lt;/participant&gt;
        &lt;/participants&gt;
        &lt;periods&gt;
          &lt;period&gt;
            &lt;period_number&gt;0&lt;/period_number&gt;
            &lt;period_description&gt;Matchups&lt;/period_description&gt;
            &lt;periodcutoff_datetimeGMT&gt;2014-12-19 11:15&lt;/periodcutoff_datetimeGMT&gt;
            &lt;period_status&gt;I&lt;/period_status&gt;
            &lt;period_update&gt;open&lt;/period_update&gt;
            &lt;spread_maximum&gt;200&lt;/spread_maximum&gt;
            &lt;moneyline_maximum&gt;100&lt;/moneyline_maximum&gt;
            &lt;total_maximum&gt;200&lt;/total_maximum&gt;
            &lt;moneyline&gt;
              &lt;moneyline_visiting&gt;116&lt;/moneyline_visiting&gt;
              &lt;moneyline_home&gt;-136&lt;/moneyline_home&gt;
            &lt;/moneyline&gt;
          &lt;/period&gt;
        &lt;/periods&gt;
        &lt;PinnacleFeedTime&gt;1418929691920&lt;/PinnacleFeedTime&gt;
      &lt;/event&gt;
    &lt;/events&gt;
  &lt;/pinnacle_line_feed&gt;
  </code></pre>
  
  <p>I have parsed the file with the code below:  </p>
  
  <pre><code>pinny_url = 'http://xml.pinnaclesports.com/pinnacleFeed.aspx?sportType=Basketball'
  
  tree = ET.parse(urllib.urlopen(pinny_url))
  root = tree.getroot()
  list = []
  
  for event in root.iter('event'):
      event_datetimeGMT = event.find('event_datetimeGMT').text
      gamenumber = event.find('gamenumber').text
      sporttype = event.find('sporttype').text
      league = event.find('league').text
      IsLive = event.find('IsLive').text
      for participants in event.iter('participants'):
          for participant in participants.iter('participant'):
              p1_name = participant.find('participant_name').text
              contestantnum  = participant.find('contestantnum').text
              rotnum = participant.find('rotnum').text
              vhd = participant.find('visiting_home_draw').text
      for periods in event.iter('periods'):
          for period in periods.iter('period'):
              period_number = period.find('period_number').text
              desc = period.find('period_description').text
              pdatetime = period.find('periodcutoff_datetimeGMT')
              status = period.find('period_status').text
              update = period.find('period_update').text
              max = period.find('spread_maximum').text
              mlmax = period.find('moneyline_maximum').text
              tot_max = period.find('total_maximum').text
              for moneyline in period.iter('moneyline'):
                  ml_vis = moneyline.find('moneyline_visiting').text
                  ml_home = moneyline.find('moneyline_home').text
  </code></pre>
  
  <p>However, I am hoping to get the nodes separated by event similar to a 2D table (as in a pandas dataframe).  However, the full xml file has multiple "event" children, some events that do not share the same nodes as above.  I am struggling quite mightily with being able to take each event node and simply create a 2d table with the tag and that value where the tag acts as the column name and the text acts as the value.  </p>
  
  <p>Up to this point, I have done the above to gauge how I might put that information into a dictionary and subsequently put a number of dictionaries into a list from which I can create a dataframe using pandas, but that has not worked out, as all attempts have required me to find and replace text to create the dxcictionaries and python has not responded well to that when attempting to subsequently create a dataframe.  I have also used a simple:</p>
  
  <pre><code>for elt in tree.iter():
    list.append("'%s': '%s'") % (elt.tag, elt.text.strip()))
  </code></pre>
  
  <p>which worked quite well in simple pulling out every single tag and the corresponding text, but I was unable to make anything of that because any attempts at finding and replacing the text to create dictionaries was no good.</p>
  
  <p>Any assistance would be greatly appreciated.</p>
  
  <p>Thank you.</p>