title: Iterparse not parsing all values in XML
body: <p>I have run the following code on a smaller file ~14MB, it works fine and converted the XML into dataframe. But when I tried some large files 500MB and 8GB I got only 88 and 98 rows respectively. I am wondering why all the values are not converted in the dataframe despite the large files having rows in a million. Image of XML data attached<a href="https://i.stack.imgur.com/WE084.png" rel="nofollow noreferrer">1</a>.</p>
  <pre><code>&lt;fcd-export xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot; xsi:noNamespaceSchemaLocation=&quot;http://sumo.dlr.de/xsd/fcd_file.xsd&quot;&gt;
      &lt;timestep time=&quot;28800.00&quot;/&gt;
      &lt;timestep time=&quot;28890.00&quot;&gt;
          &lt;vehicle id=&quot;800002&quot; x=&quot;5534.41&quot; y=&quot;3530.10&quot; angle=&quot;243.59&quot; type=&quot;HV&quot; speed=&quot;14.58&quot; pos=&quot;5.10&quot; lane=&quot;-gneE13_2&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800003&quot; x=&quot;4190.43&quot; y=&quot;3359.53&quot; angle=&quot;88.95&quot; type=&quot;HV&quot; speed=&quot;12.64&quot; pos=&quot;5.10&quot; lane=&quot;-5088609#2_0&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800017&quot; x=&quot;1972.35&quot; y=&quot;437.35&quot; angle=&quot;306.80&quot; type=&quot;HV&quot; speed=&quot;15.17&quot; pos=&quot;5.10&quot; lane=&quot;-gneE5_2&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800021&quot; x=&quot;9.34&quot; y=&quot;-1.68&quot; angle=&quot;42.62&quot; type=&quot;HV&quot; speed=&quot;13.32&quot; pos=&quot;5.10&quot; lane=&quot;-gneE6_1&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800034&quot; x=&quot;2616.22&quot; y=&quot;1599.61&quot; angle=&quot;318.00&quot; type=&quot;HV&quot; speed=&quot;14.54&quot; pos=&quot;5.10&quot; lane=&quot;-gneE2_0&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800053&quot; x=&quot;2915.19&quot; y=&quot;2618.33&quot; angle=&quot;290.01&quot; type=&quot;HV&quot; speed=&quot;16.52&quot; pos=&quot;5.10&quot; lane=&quot;-6200994#2_0&quot; slope=&quot;0.00&quot;/&gt;
      &lt;/timestep&gt;
      &lt;timestep time=&quot;28980.00&quot;&gt;
          &lt;vehicle id=&quot;800002&quot; x=&quot;5534.41&quot; y=&quot;3530.10&quot; angle=&quot;243.59&quot; type=&quot;HV&quot; speed=&quot;0.00&quot; pos=&quot;5.10&quot; lane=&quot;-gneE13_2&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800003&quot; x=&quot;4190.43&quot; y=&quot;3359.53&quot; angle=&quot;88.95&quot; type=&quot;HV&quot; speed=&quot;0.00&quot; pos=&quot;5.10&quot; lane=&quot;-5088609#2_0&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800009&quot; x=&quot;1687.49&quot; y=&quot;1643.16&quot; angle=&quot;160.62&quot; type=&quot;HV&quot; speed=&quot;6.13&quot; pos=&quot;5.10&quot; lane=&quot;-8561271#0_0&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800017&quot; x=&quot;1971.20&quot; y=&quot;438.21&quot; angle=&quot;306.80&quot; type=&quot;HV&quot; speed=&quot;0.02&quot; pos=&quot;6.53&quot; lane=&quot;-gneE5_2&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800021&quot; x=&quot;819.44&quot; y=&quot;878.63&quot; angle=&quot;42.62&quot; type=&quot;HV&quot; speed=&quot;13.32&quot; pos=&quot;1204.07&quot; lane=&quot;-gneE6_1&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800034&quot; x=&quot;2616.22&quot; y=&quot;1599.61&quot; angle=&quot;318.00&quot; type=&quot;HV&quot; speed=&quot;0.00&quot; pos=&quot;5.10&quot; lane=&quot;-gneE2_0&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800053&quot; x=&quot;2915.19&quot; y=&quot;2618.33&quot; angle=&quot;290.01&quot; type=&quot;HV&quot; speed=&quot;0.00&quot; pos=&quot;5.10&quot; lane=&quot;-6200994#2_0&quot; slope=&quot;0.00&quot;/&gt;
          &lt;vehicle id=&quot;800305&quot; x=&quot;455.31&quot; y=&quot;130.78&quot; angle=&quot;50.29&quot; type=&quot;HV&quot; speed=&quot;0.25&quot; pos=&quot;5.10&quot; lane=&quot;-gneE16_3&quot; slope=&quot;0.00&quot;/&gt;
      &lt;/timestep&gt;
  </code></pre>
  <pre><code>
      file = ET.iterparse('sample.xml', events= (&quot;start&quot;,&quot;end&quot;))
      
      context = iter(file)
      
      ev,root = next(context)
      
      data=[]
      mylevel=root.findall('timestep')
      for i in mylevel:
          timedata=i.get('time')
          
          assoc=i.findall('vehicle') 
          for j in assoc:
              iddata= j.get('id')
              speeddata = j.get('speed')
              lanedata = j.get('lane')
              
              data.append((timedata,iddata,speeddata,lanedata))            
             
      cols=['time','id','speed', 'lane']
      result = pd.DataFrame(data, columns=cols)
  
  </code></pre>