title: Good distributed general purpose filesystem in my case?
body: <p>I've been researching the idea of using distributed file system along with my dedicated servers instead of going with Amazon S3 and the results are nothing but massive headaches!</p>
  
  <p><br /><hr /></p>
  
  <p>My project have the following characteristics/requirements:</p>
  
  <ul>
  <li><p>User files are stored in <strong>dedicated servers</strong>. Each file is stored in 2 separate machines, located in different data centers (150-200 miles away from each other)</p></li>
  <li><p>I'm using Amazon RDS to host the associated mysql database (*). It's fairly compact (only hold IDs/files metadata)</p></li>
  <li><p>Files/data is around 50TB. Naturally, data does change and will definitely grow with time</p></li>
  </ul>
  
  <p><br /><hr /></p>
  
  <p><strong>My question is</strong>: is there a good general-purpose, <strong>distributed parallel fault-tolerant file system</strong> that have the following characteristics:</p>
  
  <ul>
  <li><p>Stable &amp; reasonably fast (upload/download)</p></li>
  <li><p>Fairly easy to setup &amp; maintain</p></li>
  <li><p>Handle data storage so that I <strong>only</strong> have to care about removing/adding new servers if the need arise (ie. add new servers to the filesystem's server pool by editing a simple config, or something like that)</p></li>
  </ul>
  
  <p>I've read about OpenStack, GlusterFS, MogileFS, XtreemFS, etc...but the more I read, the more I get confused!</p>
  
  <p>(*) Yes, I realize the contradiction. Cost-wise it does make sense to host the database on RDS. But storing (up to) 50TB of users files on amazon is way too expensive compared to using dedicated servers (provided it's good enough).</p>
  
  <p><strong>PS.</strong> my app isn't live yet, so I'm open to suggestion if someone have a good idea that fits well in my case.</p>
  
  <p><br /><hr /></p>
  
  <p><strong>EDIT</strong> I'm <strong>not</strong> trying to make a S3 clone, I just need to use an existing hosting infrastructure to build small-scale cloud solution, my question is about finding the right distributed file system to handle/automate this.</p>