title: Scalability of open source XML Databases
body: <p>We are looking to develop a reporting application that reports on data stored in a large number of XML files.  ~3,000,000 files ranging in size from 7KB to 5MB (Each file conforms to the same schema). I’m guessing that there will be about around 200GB of XML.   I’m looking at a number of open source XML databases (Sedna, BaseX and eXist-db) and I’m not sure how well these systems will scale, I read a comparison of these three database <a href="http://share.aweta.nl/weblinks/?id=f29c21d4897f78948b91f03172341b7b&amp;filename=XML%20Database%20Comparison.pdf" rel="noreferrer">here</a>. Which is where my concerns of scalability originated from. </p>
  
  <p>Some details regarding what we want to do are: We won’t be changing the data in any of the XML files and new files will be added daily. Since we are concerned with reporting query performance is important to us, and the time it takes to add and index new files isn’t a high priority for us.</p>
  
  <p>I’m wondering if anyone has experience using these systems at similar scales?  I’ve looked at the BaseX <a href="http://docs.basex.org/wiki/Statistics" rel="noreferrer">statistics page</a> and see some fairly large XML instances but no mention of performance. </p>
  
  <p>We don’t require an open source product and the MarkLogic system looks like it can fit the bill nicely, but I’m curious what’s been done with open source products. </p>