title: How to handle a datacenter outage in a mutil-datacenter Cassandra Cluster
body: <p>Our application is running in a Cassandra Cluster of six nodes with two data centers.</p>
  
  <p><strong>Cluster information:</strong></p>
  
  <p><em>Cassandra version</em>     : 2.0.3</p>
  
  <p><em>Snitch</em>                          : GossipingPropertyFileSnith</p>
  
  <p><em>Partitioner</em>                       : Murmur3Partitioner</p>
  
  <p>Each dc has three nodes.</p>
  
  <p>Each dc has replication factor as 2.</p>
  
  <p>Each node uses num_vnodes = 256. (all are virtual nodes)</p>
  
  <p>DC1 is a live dc(local dc) which serves the data to the users currently. DC2 is just a back up dc(remote dc) which does not serve any data to the users. As we are planning for maintenance operations in DC1 alone, we are going to make remote dc DC2 to serve the users during the maintenance period. </p>
  
  <p>During the outage, entire DC1 might be down for few days. Once the maintenance is done, we will again make DC1 to serve the data and DC2 for backing up. So we need to have up-to-date data in DC1 after outage. Our application will deal with huge data (few GBs) during the outage.</p>
  
  <p><strong>Before making DC1 down,</strong></p>
  
  <p>1) What are all the things (like commit-log settings,etc) need to be taken care in DC1 nodes </p>
  
  <p>2) What are all the things (like hinted-handoff settings,etc) need to be taken care in DC2 nodes </p>
  
  <p><strong>During the outage,</strong></p>
  
  <p>3) When the entire DC1 is down, where the hints will be written (in any of the nodes of DC2?) and how to handle those hints?</p>
  
  <p><strong>After DC1 is up,</strong></p>
  
  <p>4) During the outage, replication might fail in DC1 nodes. How can we make/repair the DC1 with up-to-date data efficiently using DC2?</p>