title: How to remove duplicate elements from 2 XML files?
body: <p>I have a XML that I am parsing to my database, this XML size is about 10MB, and it contains few millions of elements.</p>
  
  <p>Every 30-60 seconds its being updated, but only several elements, not all of it.</p>
  
  <p>In order to parse it in the fastest way to the database, I developed a function to remove the duplicate elements from the 2 XML files, here's the code:</p>
  
  <pre><code>XDocument doc2 = XDocument.Parse(tempDoc.ToString());
  
  var doc1 = new XDocument();
  try
  {
  doc1 = XDocument.Load(bookieName + ".xml");
  }
  catch
  { }
  
  try
  {
  var dict1 = doc1.Descendants("event").Select(el =&gt; el.ToString()).ToList();
  var dict2 = doc1.Descendants("event").Select(el =&gt; el.ToString()).ToList();
  
  foreach (var elem in dict1)
  {
      if (dict2.Contains(elem))
      {
      if (dict2.Find(x =&gt; x == elem).ToString() == dict1.Find(x =&gt; x == elem).ToString())
      {
          doc2.Descendants("event").Where(x =&gt; x.ToString() == elem).Remove();
      }
      }
  }
  }
  catch (Exception e)
  {
  Console.WriteLine(e.Message);
  }
  
  return doc2;
  </code></pre>
  
  <p>The problem is, it works great for small XML files, but for large XML files (10MB) it takes 100% CPU for 2-5 minutes.</p>
  
  <p>Do you have any idea how can I improve it and make it better?</p>
  
  <p>I am kinda hopeless.</p>
  
  <p>Thanks!</p>