title: Should I take live migrations into account when tracking measuring performance of Amazon EC2 instances?
body: <p>One of our applications is running on a number of EC2 instances, and we dynamically add/remove instances based on load. We want to try to ensure that performance variation across the images stays within a certain (configurable) bound. </p>
  
  <p>As has been demonstrated by various studies, especially [1], simply starting the same type of instance is often not sufficient if the acceptable variation is small:</p>
  
  <p>"We demonstrated that the CPU and disk I/O performance of small instances are relatively stable from the perspective of a long-running periods. However, the performance behavior of multiple “identical" small instances is very heterogeneous."</p>
  
  <p>As a result, we are now currently starting a few more instances then required, running a benchmark and keeping only those whose performance is acceptable. There's an assumption here that, if the instance performs adequately during the benchmark, its performance will be relatively stable.</p>
  
  <p>This appears to be mainly due to the fact that, once an instance is started, it remains on the same underlying host until stopped or terminated (see e.g. [2]). However, there have been recent reports that EC2 has changed parts of the underlying Xen infrastructure and can now support live migration (internally, i.e. not exposed to the user) [3].</p>
  
  <p>"You can infer with relative confidence that that means they’ve fully divorced the instance from the hardware and can therefore now live-migrate instances."</p>
  
  <p>If Amazon <em>does</em> start to live-migrate instances across hosts, then of course benchmarking-on-startup isn't sufficient. So the question is: should we assume that Amazon could or will be live-migrating instances "under the covers" in the near future?</p>
  
  <p>Thanks</p>
  
  <p>Andrew</p>
  
  <p>[1] "EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications", <a href="http://www.globule.org/publi/EPARPSOA_nfpsla2009.pdf" rel="nofollow">http://www.globule.org/publi/EPARPSOA_nfpsla2009.pdf</a></p>
  
  <p>[2] Response of "JasonM@AWS", <a href="https://forums.aws.amazon.com/thread.jspa?messageID=198800&#198800" rel="nofollow">https://forums.aws.amazon.com/thread.jspa?messageID=198800&#198800</a></p>
  
  <p>[3] <a href="http://default.io/2010/09/to-do-learn-ec2-and-ubuntu-at-the-same-time/" rel="nofollow">http://default.io/2010/09/to-do-learn-ec2-and-ubuntu-at-the-same-time/</a></p>