title: Efficient removal of XML elements using Python
body: <p>I am trying to efficiently edit reasonably large XML files (usually 100-500MB but up to 1GB) in size to remove all occurrences of an element which do not contain an attribute with a given value. I am looking for the most efficient way of performing this in terms of speed, whilst also not loading large amounts of data into memory since this is an issue for larger files. </p>
  
  <p>Using example XML, the structure is along the lines of the following, where the parent element may be nested within each other any number of times.</p>
  
  <pre><code>&lt;root&gt;
  &lt;parent&gt;
      &lt;child id="c1"&gt;
          &lt;content /&gt;
      &lt;/child&gt;
      &lt;child id="c2"&gt;
          &lt;content /&gt;
      &lt;/child&gt;
  &lt;/parent&gt;
  &lt;parent&gt;
      &lt;parent&gt;
          &lt;child id="c3"&gt;
              &lt;content /&gt;
          &lt;/child&gt;
      &lt;/parent&gt;
  &lt;/parent&gt;
  &lt;/root&gt;
  </code></pre>
  
  <p>Using the above example XML, I am trying to remove all child elements where the ID doesn't equal "c1" to give an outcome of:</p>
  
  <pre><code>&lt;root&gt;
  &lt;parent&gt;
      &lt;child id="c1"&gt;
          &lt;content /&gt;
      &lt;/child&gt;
  &lt;/parent&gt;
  &lt;parent&gt;
      &lt;parent /&gt;
  &lt;/parent&gt;
  &lt;/root&gt;
  </code></pre>
  
  <p>The most efficient method I have come up with so far is using cElementTree iterparse:</p>
  
  <pre><code>import xml.etree.cElementTree as ET
  
  xml_source = 'xml file location'
  xml_output = 'xml output file location'
  
  context = ET.iterparse(xml_source, events=("start", "end"))
  context = iter(context)
  
  event, root = context.next()
  
  for event, elem in context:
      if event == 'end' and elem.tag == 'child' and elem.attrib['id'] != 'c1':
          elem.clear()
  
  ET.ElementTree(root).write(xml_output)
  </code></pre>
  
  <p>The above will handle a test file 100MB in size in around 10 seconds, is there a more efficient way of achieving this?</p>