title: Architecture - Selecting a NoSQL for .NET application
body: <p>This question is about selecting the "right" type of NoSQL database, and I hope maybe even discuss specific ones and why they fit, according to some requirements/use-cases that I will list below together with the traditional RDBMS solution that is currently in place. It is a little long but I think that any discussion on this topic can be really beneficial to people that are trying to learn the new paradigm(s). There are many discussion about NoSQL but from what I have seen - most of them are high level and don't give enough insight for newbies.</p>
  
  <p>So, here it comes:</p>
  
  <p>I've been developing against traditional RDBMS/SQL systems for most of my programming career (15 years) and have good experience with it. Lately, there is a big buzz about NoSQL and how useful it is - so I am interested to understand how it can be beneficial. The system I describe is a bit more complex than the average TODO or Calender example that I've seen and thus, can make good discussion.</p>
  
  <p>The system is related to cellular networks which are relatively complex - there are about 300  "classes" in such network (and a "full deployment" can have several network together and can grow up to 1000 and more classes) with varying number of instances (100,000 - 10s) for each. This are loaded each days (sometimes a few times a day) to a database in order to drive the system. The relations between the classes are either containment or "usage". The domain is changing relatively fast (about 3 month between software updates of the network, each usually means adding parameters to existing classes and adding a few (10-20) new classes).</p>
  
  <p>The usage (use cases) of the system was as follow:
  0. parse the data (into data-containers hierarchy) and load it to a relational database (usually from XML files of around 2GB)</p>
  
  <ol>
  <li>viewing properties (like "select field1, field2 from table1 where ids in ()" and viewing in a table format</li>
  <li>tracking changes (what change between today and yesterday - parameters who's value has changed and added/removed instances</li>
  <li>checking business rules:
  <ul>
  <li>it can be simple (SELECT idField1...idFieldN, paramValue FROM table where paramValue&lt;>default"</li>
  <li>or more complex - checking on relations - e.g. number of children of type x etc</li>
  </ul></li>
  <li>retrieving all hierarchy of class - select specific class instance(s), its children and sometimes, classes that are used by the instance or it children</li>
  <li>make changes to the class instances and push back into the network (then see it was indeed executed - validation of the changes). This usually required to generate some XML file based on the hierarchy of the classes.</li>
  </ol>
  
  <p>In the RDBMS solution, to overcome these requirements I mapped the data into relational tables (a class for each) and then held a metadata and relations dictionary. In addition, for data retrieval tasks created a general data container (class type name + key-value (or values)) or use DataTables that could be merged into views or files. </p>
  
  <p>This architecture (platform) meant that on an upgrade all I had to do is update/create the tables (alter/create table) and update the metadata and relations - the rest of the code was "generic" and driven by the metadata. The only exception was for (4) above which sometimes required me to hard code (add children to data retrieval hierarchy) though eventually I generalized this processes as well (hierarchical data retrieval-get child element based on id of parent and so on down the hierarchy).</p>
  
  <p>The system works well in most cases but sometimes was too slow (especially in 4). The slowness was related to the retrieval of data from the DB but only in some deployment and it maybe related to poor maintenance or insufficient hardware (or bad programming but then, why it works well in other deployments?-)</p>
  
  <p>I will add that since the domain is a network, each instance has a Distinct Name - usually consisting of it's hierarchy (the instance and it's parent e.g. "Node=ER222,Subrack=3,Slot=5" or "Node=ER222,Equipment=1,Sector=2,Carrier=C2") and the hierarchy of each class is generally the same (though some classes can appear in several hierarchy (e.g. have different ancestors)</p>
  
  <p>Usually there is not much load on the system - maybe up to as much as 50 active users but usually much less. In a larger network this can maybe grow up to 300-400 users.</p>
  
  <p>Now I want to develop a system with similar requirement and am considering what advantages NoSQL may give:</p>
  
  <ul>
  <li>I read that for dynamic-schema or schema-less NoSQL is a natural
  choisce.  </li>
  <li>I read the graph databases are good for modeling "network"
  (or network like) so maybe that could be a solution (node=class, edge
  = containment or usage (having attribute on the edge)).  </li>
  <li>Maybe use some document database and keep the XML only partially parsed and access it by the hierarchy? 
  <ul>
  <li>How do I go about selecting specific fields from specific classes - do I have to generate gruesome XPath queries for that?</li>
  </ul></li>
  <li>Maybe an object database?
  <ul>
  <li>but then - do I have to keep a (bloated) model of 1000 or more POCOs? How easy it will be to serialize/deserialize?</li>
  </ul></li>
  </ul>
  
  <p>In addition to the above, I am developing with .NET technologies so if anyone has specific ideas - better ones that fit into this ecosystem or at least can be developed with .NET (e.g. REST/THRIFT interface and matching .NET API)</p>
  
  <p>If you read that far - I appreciate it a lot and if you care to join in- even more so ;-)</p>