title: DB candidate as CouchDB/Schema replacement
body: <p>The idea is to redesign data structure and/or change DB.
  I just started to review this project and plan to start optimization from this one.</p>
  
  <p>Currently i have CouchDb with about <strong>80GB</strong> of document data, around <strong>30M</strong> records.
  From that subset for the most of documents properties like <strong>id, group_id, location, type</strong> can be considered as generic, but unfortunately for now such are even stored with different property naming around the set. Also a lot of deeply nested can be found.</p>
  
  <p>Structure isn't hardly defined, that's why NoSQL db was selected way before some picture was seen.</p>
  
  <p>Data is calculated and populated in DB in a separate Job on powerful cluster. This isn't done too often. From that perspective i can conclude that general write/update performance isn't very important. Also size decrease would be great, but isn't most important. There are only like 1-10 active customers at a time.
  Actually read performance with various filtering/grouping etc is most important.
  But no heavy summary calculations should be done, this one is already done while population. </p>
  
  <p>This one is a data analytical tool for displaying compare and other reports to quality engineers and data analyst, so they can browse the results, group them or filter from the Web UI. </p>
  
  <p>Now such tasks like searching a subset of document properties for a text isn't possible due to performance.</p>
  
  <p>For sure i've done some initial investigations(like <a href="http://www.datastax.com/wp-content/themes/datastax-2014-08/files/NoSQL_Benchmarks_EndPoint.pdf" rel="nofollow">http://www.datastax.com/wp-content/themes/datastax-2014-08/files/NoSQL_Benchmarks_EndPoint.pdf</a>) and it looks Cassandra seems to be good choice among NoSql.</p>
  
  <p>Also it's quite interesting trying to port this data into the new PostgreSQl.</p>
  
  <p>Any ideas would be highly appreciated :-)</p>