abstract: The problem of initializing phase in a quantum computing system is considered. The initialization of phases is a problem when the system is initially present in a superposition state as well as in the application of the quantum gate transformations, since each gate will introduce phase uncertainty. The accumulation of these random phases will reduce the effectiveness of the recently proposed quantum computing schemes. The paper also presents general observations on the nonlocal nature of quantum errors and the expected performance of the proposed quantum error-correction codes that are based on the assumption that the errors are either bit-flip or phase-flip or both. It is argued that these codes cannot directly solve the initialization problem of quantum computing.
  .
  *Keywords*: Quantum computing, error-correction, initialization
author: Subhash Kak
Department of Electrical & Computer Engineering
Louisiana State University
Baton Rouge, LA 70803-5901; `email@example.com`
date: November 29, 1999
title: The Initialization Problem in Quantum Computing

## Foundations of Physics, vol 29, 1999, pp. 267-279

# Introduction

A quantum state is undetermined with respect to its phase. This indeterminacy is in principle irremovable. The uncertainty of phase, together with superposition, is responsible for the power of quantum mechanics. It also compels us to speak of information in *positivist* terms—with respect to an observation rather than in an absolute sense.

Since phase indeterminacy is fundamental to quantum description, it is relevant to examine its implications for quantum computation. This indeterminacy can manifest itself in a variety of ways due to the interaction with the environment or while initializing the quantum register. Quantum computing algorithms assume that the state of the quantum register has its phase uncertainty lumped together, so that it can be ignored. This is true enough in certain idealized state preparations. But more realistic situations may not permit it to be lumped together.

Effects of decoherence in the implementation of quantum computation have been widely discussed in the literature as one of its main drawbacks. Random phase shifts, without disentaglement of the states, can also cause serious problems in an ongoing quantum computation. These are unitary transformations of the form:

$$\frac{1}{\sqrt {||a||^2 + ||b||^2}} \left[ \begin{array}{cc}
                                  a^* & b^* \\
                                  b & -a \\
                               \end{array} \right]$$ where $a^*$ and $b^*$ contain unknown phase angles.

Recent quantum computation algorithms use a method of increasing the amplitude of a marked solution state at the expense of unmarked states. This is achieved by changing the difference in the phase angles of the marked and the unmarked states. But injection of random phases makes it impossible to perform search that will exploit quantum parallelism.

At the implementation level, the representation of a unitary transformation in terms of a sequence of small-degree gates (such as 2-bit gates) will introduce random phase shifts at each gate that will have an effect similar to random phases in a register. In other words, the current conception of quantum computers appears to be unsuitable in terms of implementation.

# State preparation

A quantum register of length $n$ is postulated where all the superpositions of the $N=2^n$ basis states exist with the amplitudes:

$$(\frac{1}{\sqrt N},\frac{1}{\sqrt N},\frac{1}{\sqrt N} . . .\frac{1}{\sqrt N})$$

The idea in the initialization of the quantum register is to place all the $N=2^n$ states in a superposition where each basis state is equally probable. But how is this done? By placing a bit, say a $0$, in each cell. Now, the following transformation

$$M = \frac{1}{\sqrt 2} \left[ \begin{array}{cc}
                                  1 & 1 \\
                                  1 & -1 \\
                               \end{array} \right]$$ is applied to each bit, transforming it into the superposition with the amplitudes $(
\frac{1}{\sqrt 2}, \frac{1}{\sqrt 2})$.

But this would be true only to within an unknown phase angle. Strictly speaking, the state of the cell should be written as:

$$|\phi\rangle = \frac{e^{i \theta }}{\sqrt 2}( |0\rangle +  |1\rangle)$$ where $\theta$ is the unknown phase of the initial $0$.

Such a preparation for each qubit leads to the lumping together of the uncertainty for the state of the register. Here we can imagine that photons have been passed through a horizontal polarizing filter and then rotated by the transformation $M$ to produce superposition qubits.

But to consider this problem from a less idealistic perspective, it should be remembered that the object that carries the qubit, be it a photon, an electron, or an atom or a molecule with a certain spin state, is already physically present at its location. Given that fact, the initialization procedure is to let the object relax to the superposition state which in its most general form will be:

$$|\phi\rangle = \frac{e^{i \theta_{11} }}{\sqrt 2} |0\rangle +
 \frac{e^{i \theta_{12} }}{\sqrt 2} |1\rangle$$

The state of the quantum register will then be:

$$|\phi\rangle = \frac{e^{i \theta_{1} }}{\sqrt N} |0...0\rangle +
 \frac{e^{i \theta_{2} }}{\sqrt N} |0...1\rangle + . . . +
 \frac{e^{i \theta_{N} }}{\sqrt N} |1...1\rangle$$

Although each of the $2^n$ states has the same probability, the associated phases are unknown and so it is impossible to use a method of amplitude amplification on any marked state. Phase rotation for a case where the phases are randomly distributed will be meaningless.

# Quantum gates

It is normal to speak of the phase function with the state of the quantum system, but this can also be expressed, equivalently, in terms of an arbitrary phase associated with the unitary operator because, operationally, from the point of view of a measurement, these two are indistinguishable. Clearly, the problem of the initialization of the quantum register will have a parallel in the initialization of the apparatus used to implement unitary transformations.

The quantum computers implements the time-evolution operator $U$, that represents the transformation on the data in the register, in terms of smaller gates. For example, DiVincenzo showed that two-bit gates are universal for quantum computation. This was done by showing that appropriate sequence of two-bit gates can realize Deutsch's three-gates that implement the Toffoli reversible gates.

But the unitary transformation with each gate, in itself, is associated with an unknown phase, and these values will migrate in the direct product operation used to construct the larger gates. In other words, the realization of the system unitary matrix in terms of the small gates will be correct only in the absence of the random phases. In the recursive development of the S-matrices for the various gates, Deutsch failed to include the unknown phase with the embedded S-terms, assuming thereby that the gates were initialized.

# Random phase shifts and decoherence

Implementations of quantum computing based on trapped ions, quantum dots, cavity-QED, and nuclear magnetic resonance (NMR) are being investigated. Here it is assumed that there are no randomized phases in the initialized register. But we must consider the issue of decoherence that leads to a decay of the superposition of states to a particular base state due to an interaction with the environment. The decoherence time can vary from $10^{-12}$ sec for electron-hole excitation in the bulk of a semiconductor to $10^4$ sec for nuclear spin on a paramagnetic atom. If $t_d$ is the typical decoherence time, the decoherence characteristics for a single qubit are proportional to $e^{-t/t_d}$.

For multiple qubits one must multiply the individual characteristics. For a quantum register of $n$ qubits, the decoherence characteristics are given by

$$e^{-tn/t_d} .$$

In other words, the effective decoherence time decreases linearly with the length of the register.

Decoherence may be viewed as a decay in the off-diagonal elements of the density matrix representation of the state of the register. But in operational terms, the process is a cumulative effect of random phase shifts introduced by the interaction with the atoms of the environment. Ultimately, the qubit object falls in one of the basis states in equilibrium with the state function of the environment.

In this perspective, the equilibrium may be viewed to be the end result of a walk executed by the phase of each qubit within the energy state basin to its least value. But since the qubits are physically isolated to the extent possible, one may take this walk to be a random one. If the step size in this walk is $s$, associated with a characteristic time of $\tau$, then after $m\tau$ will be $m s^2$.

Since, the distribution of the random walk can be approximated by the Gaussian function, a Gaussian error with a linearly increasing variance will characterize the departure from the desired values of the phase angles.

To stress why the knowledge of relative phases is important consider Grover's quantum search algorithm, where a certain transformation is applied to the state which computes the property that the database item being searched uniquely satisfies, marking that state in the process, further generating transformed states in superposition. Next, is the procedure that increases the amplitude of the marked state progressively: the phase angle of the marked state is rotated through $\pi$ radians and the diffusion transform $D$ applied as follows:

$$D_{ij} = \frac{2}{N} ~~ if  ~~i \neq j ~~ \&
D_{ii} = -1 + \frac{2}{N}.$$

This process is repeated a total of about $\frac{\pi}{4}\sqrt N$ times after which the state is measured when it is found in the marked state with probability close to 1, thus allowing us to find the database item in about $\sqrt N$ steps compared to the average of $\frac{N}{2}$ steps in a classical algorithm.

In this algorithm, an error of $\epsilon$ in the phase of the marked state will cause a corresponding error of $\frac{1}{\sqrt N}(-\epsilon + \frac{2\epsilon}{N})$ in the amplitude of the marked state, if it is assumed that the errors in the phases of the other states cancel out.

When $N$ is large, the error in amplitude will be $-\epsilon/\sqrt N$ in each step, but this will progressively increase in subsequent steps.

# Quantum error correcting codes

In a classical information system the basic error is represented by a $0$ becoming a $1$ or vice versa. The characterization of such errors is in terms of an error rate, $\epsilon$, associated with such flips. The correction of such errors is achieved by appending check bits to a block of information bits. The redundancy provided by the check bits can be exploited to determine the location of errors using the method of syndrome decoding. These codes are characterized by a certain capacity for error-correction per block. Errors at a rate less than the capacity of the code are *completely* corrected.

Now let us look at a quantum system. Consider a single cell in a quantum register. The error here can be due to a random unitary transformation or by entanglement with the environment. These errors cannot be defined in a graded sense because of the group property of unitary matrices and the many different ways the entanglements can be expressed. Let us consider just the first type of error, namely that of random unitary transformation. If the qubit is the state $| 0\rangle$, it can become $a |0\rangle + b | 1 \rangle$. Likewise, the state $| 00\rangle$ can become $a | 00\rangle + b | 01 \rangle + c | 10 \rangle + d | 11\rangle$. In the initialization of the qubit a similar error can occur. If the initialization process consists of collapsing a random qubit to the basis state $| 0\rangle$, the definition of the basis direction can itself have a small error associated with it. This error is analog and so, unlike error in classical digital systems, it cannot be controlled. In almost all cases, therefore, the qubits will have entangled states, although the level of entanglement may be very low.

From another perspective, classical error-correction codes map the information bits into codewords in a higher dimensional space so that if just a few errors occur in the codeword, their location can, upon decoding, be identified. This identification is possible because the errors perturb the codewords, *locally*, within small spheres. Quantum errors, on the other hand, perturb the information bits, in a *nonlocal* sense, to a superposition of many states so the concept of controlling all errors by using a higher dimensional codeword space cannot be directly applied.

According to the positivist understanding of quantum mechanics, it is essential to speak from the point of view of the observer and not ask about any intrinsic information in a quantum state. Let's consider, therefore, the representation of errors by means of particles in a register of $N$ states.

We could consider errors to be equivalent to either $n$ bosons or fermions. Bosons, in a superpositional state, follow the Bose-Einstein statistics. The probability of each pattern will be given by

$$\frac{1}{\left( \begin{array}{c}
                N + n - 1 \\ n
                \end{array}
                  \right) }.$$

So if there are 3 states and 1 error particle, we can only distinguish between 3 states: $00,~01~or~10,~11$. Each of these will have a probability of $\frac{1}{3}$. To the extent this distribution departs from that of classical mechanics, it represents nonlocality at work.

If the particles are fermions, then they are indistinguishable, and with $n$ error objects in $N$ cells, we have each with the probability

$$\frac{1}{\left( \begin{array}{c}
                N  \\ n
                \end{array}
                  \right) }.$$

If states and particles have been identified, these statistics will be manifested by a group of particles. If the cells are isolated then their histories cannot be described by a single unitary transformation.

Like the particles, the errors will also be subject to the same statistics. These statistics imply that the errors will not be independent, an assumption that is basic to the error-correction schemes examined in the literature.

To summarize, important characteristics of quantum errors that must be considered are *component proliferation*, *nonlocal effects* and *amplitude error*. All of these have no parallel in the classical case. Furthermore, quantum errors are analog and so the system cannot be shielded below a given error rate. Such shielding is possible for classical digital systems.

We know that a computation need not require any expenditure of energy if it is cast in the form of a reversible process. A computation which is not reversible must involve energy dissipation. Considering conservation of information+energy to be a fundamental principle, a correction of random errors in the qubits by unitary transformations, without any expenditure of energy, violates this principle.

Can we devise error-correction coding for quantum systems? To examine this, consider the problem of protein-folding, believed to be NP-complete, which is, nevertheless, solved efficiently by Nature. If a quantum process is at the basis of this amazing result, then it is almost certain that reliable or fault-tolerant quantum computing must exist but, paying heed to the above-mentioned conservation law, it appears such computing will require some lossy operations.

# Representing quantum errors

Every unitary matrix can be transformed by a suitable unitary matrix into a diagonal matrix with all its elements of unit modulus. The reverse also being true, quantum errors can play havoc.

The general unitary transformation representing errors for a qubit is:

$$\frac{1}{\sqrt {||e_1||^2 + ||e_2||^2}} \left[ \begin{array}{cc}
                                  e_1^* & e_2^* \\
                                  e_2 & -e_1 \\
                               \end{array} \right] .$$

These errors ultimately change the probabilities of the qubit being decoded as a $0$ and as a $1$. From the point of view of the user, when the quantum state has collapsed to one of its basis states, it is correct to speak of an error rate. But such an error rate cannot be directly applied to the quantum state itself.

Unlike the classical digital case, quantum errors cannot be completely eliminated because they are essentially analog in nature.

The unitary matrix (11) represents an infinite number of cases of error. The error process is an analog process, and so, in general, such errors cannot be corrected. From the point of view of the qubits, it is a nonlocal process.

If it is assumed that the error process can be represented by a small rotation and the initial state is either a $0$ or a $1$, then this rotation will generate a superposition of the two states but the relative amplitudes will be different and these could be exploited in some specific situations to determine the starting state. But, obviously, such examples represent trivial cases.

The error process may be usefully represented by a process of quantum diffusions and phase rotations.

Shor showed how the decoherence in a qubit could be corrected by a system of triple redundancy coding where each qubit is encoded into nine qubits as follows:

$$|0\rangle \rightarrow \frac{1}{2\sqrt 2}
( |000\rangle + | 111\rangle )
( |000\rangle + | 111\rangle )
( |000\rangle + | 111\rangle )$$,

$$|1\rangle \rightarrow \frac{1}{2\sqrt 2}
( |000\rangle - | 111\rangle )
( |000\rangle - | 111\rangle )
( |000\rangle - | 111\rangle ).$$

Shor considers the decoherence process to be one where a qubit decays into a weighted amplitude superposition of its basis states. In parallel to the assumption of independence of noise in classical information theory, Shor assumes that only one qubit out of the total of nine decoheres. Using a Bell basis, Shor then shows that one can determine the error and correct it.

But this system does not work if more than one qubit is in error. Quantum error is analog, so each qubit will be in some error and so this scheme will, in practice, not be useful in *completely* eliminating errors.

The question of decoherence, or error, must be considered as a function of time. One may use the exponential function $\lambda e^{-\lambda t}$ as a measure of the decoherence probability of the amplitude of the qubit. The measure of decoherence that has taken place by time $t$ will then be given by the probability, $p_t$:

$$p_t = 1 - \lambda e^{-\lambda t}.$$

In other words, by time $t$, the amplitude of the qubit would have decayed to a fraction $(1 - \lambda e^{-\lambda t})$ of its original value. At any time $t$, there is a $100 \%$ chance that the probability amplitude of the initial state will be a fraction $\alpha_k < 1$ of the initial amplitude.

If we consider a rotation error in each qubit through angle $\theta$, there exists some $\theta_k$ so that the probability

$$Prob ( \theta > \theta_k) \rightarrow 1.$$

This means that we cannot represent the qubit error probability by an assumed value $p$ as was done by Shor in analogy with the classical case. In other words, there can be no guarantee of eliminating decoherence.

# Recently proposed error-correction codes

The recently proposed models of quantum error-correction codes assume that the error in the qubit state $a |0\rangle + b | 1 \rangle$ can be either a bit flip $|0 \rangle \leftrightarrow | 1 \rangle$, a phase flip between the relative phases of $| 0\rangle$ and $| 1 \rangle$, or both .

In other words, the errors are supposed to take the pair of amplitudes $(a,b)$ to either $(b,a)$, $(a, -b)$, or $(-b,a)$.

But these three cases represent a vanishingly small subset of all the random unitary transformations associated with arbitrary error. These are just three of the infinity of rotations and diffusions that the qubit can be subject to. The assumed errors, which are all local, do not, therefore, constitute a distinguished set on any physical basis.

In one proposed error-correction code, each of the states $|0\rangle$ or $| 1 \rangle$ is represented by a 7-qubit code, where the strings of the codewords represent the codewords of the single-error correcting Hamming code, the details of which we don't need to get into here. The code for $| 0 \rangle$ has an even number of $1$s and the code for $|1 \rangle$ has an odd number of $1$s.

$$| 0\rangle_{code} = \frac{1}{\sqrt8} (|0000000\rangle + |0001111\rangle+
|0110011\rangle + |0111100\rangle\\$$

$$+ |1010101\rangle
+|1011010\rangle +| 1100110\rangle + |1101001\rangle),$$

$$|1\rangle_{code} = \frac{1}{\sqrt8} (|1111111\rangle + |1110000\rangle+
|1001100\rangle + |1000011\rangle \\$$

$$+ |0101010\rangle
+|0100101\rangle +| 0011001\rangle + |0010110\rangle).$$

As mentioned before, the errors are assumed to be either in terms of phase-flips or bit-flips. Now further ancilla bits— three in total— are augmented that compute the syndrome values. The bit-flips, so long as limited to one in each group, can be computed directly from the syndrome. The phase-flips are likewise computed, but only after a change of the bases has been performed.

Without going into the details of these steps, which are a straightforward generalization of classical error correction theory, it is clear that the assumption of single phase and bit-flips is completely artificial.

In reality, errors in the 7-qubit words will generate an superposition state of 128 sequences, rather than the 16 sequences of equations (15) and (16), together with 16 other sequences of one-bit errors, where the errors in the amplitudes are limited to the phase-flips mentioned above. *All kinds of bit-flips*, as well as modifications of the amplitudes will be a part of the quantum state.

We can represent the state, with the appropriate phase shifts associated with each of the 128 component states, as follows:

$$|\phi\rangle = e^{i \theta_{1} } a_1 |0000000\rangle +
 e^{i \theta_{2} } a_2 |0000001\rangle + . . . +
 e^{i \theta_{N} } a_N |1111111\rangle)$$

While the amplitudes of the newly generated components will be small, they would, nevertheless, have a non-zero error probability. These components, cannot be corrected by the code and will, therefore, contribute to an residual error probability.

The amplitudes implied by (17) will, for the 16 sequences of the original codeword after the error has enlarged the set, be somewhat different from the original values. So if we speak just of the 16 sequences the amplitudes cannot be preserved without error.

Furthermore, the phase errors in (17) cannot be corrected. These phases are of crucial importance in many recent quantum algorithms.

It is normally understood that in classical systems if error rate is smaller than a certain value, the error-correction system will correct it. In the quantum error-correction systems, this important criterion is violated. Only certain specific errors are corrected, others even if smaller, are not.

In summary, the proposed models are based on a local error model while real errors are nonlocal where we must consider the issues of component proliferation and amplitude errors. These codes are not capable of completely correcting small errors that cause new component states to be created.

# The sensitivity to errors

The nonlocal nature of the quantum errors is seen clearly in the sensitivity characteristics of these errors.

Consider that some data sets related to a problem are being simultaneously processed by a quantum machine. Assume that by some process of phase switching and diffusion the amplitude of the desired solution out of the entire set is slowly increased at the expense of the others. Nearing the end of the computation, the sensitivity of the computations to errors will increase dramatically, because the errors will, proportionately, increase for the smaller amplitudes. To see it differently, it will be much harder to reverse the computation if the change in the amplitude or phase is proportionally greater.

This means that the "cost" of quantum error-correction will depend on the state of the computing system. Even in the absence of errors, the sensitivity will change as the state evolves, a result, no doubt, due to the nonlocal nature of quantum errors. These errors can be considered to be present at the stage of state preparation and through the continuing interaction with the environment and also due to the errors in the applied transformations to the data. In addition, there may exist nonlocal correlations of qubits with those in the environment. The effect of such correlations will be unpredictable.

Quantum errors cannot be localized. For example, when speaking of rotation errors, there always exists some $\theta_k > 0$ so that $prob (\theta > \theta_k) \rightarrow 1$.

When doing numerical calculations on a computer, it is essential to have an operating regime that provides reliable, fault-tolerant processing. Such regimes exist in classical computing. But the models currently under examiniation for quantum computing cannot eliminate errors completely. The method of syndrome decoding, adapted from the theory of classical error-correcting codes, appears not to be the answer to the problem of fault-tolerant quantum computing. New approaches to error-correction may be needed.

# Conclusions

The undetermined phase of a quantum state can be seen, equivalently, in an undetermined phase associated with each unitary operator. Normally, this has no significance because the usual representations deal with the entire system and so the phase is effectively a lumped term that has no observational value. In considering a unitary transformation as being built out of smaller blocks, the phase cannot be ignored. In other words, there is no simple way we can effectively "initialize" each quantum gate.

If one did not concern oneself with the question of the realization of the gates, assuming that the system unitary transformation will be somehow carried out, one still has a difficulty with the random phases in the component states of a quantum register. Given these random phases, one cannot manipulate the amplitudes to increase the value for a marked state as is required in the search problem. If the random phases exist in the initialized register, computations exploiting quantum superposition cannot be performed. If the randomization doesn't exist in the initialized register and is forced upon the computation in the later stages, then this might shorten the time range where useful computations can be performed even more than by decoherence.

We don't know of any simple method to correct for the random phase errors by the use of the proposed quantum error correction codes because these errors will not necessarily be within bounds. Besides, the error-correction systems will be plagued with the same random phase problems that apply to other quantum gates and registers. As mentioned before, it is nonlocality, related both to the evolution of the quantum information system and errors, that makes it difficult for syndrome decoding to work.

How should error-correction be defined then? Perhaps through a system akin to associative learning in spin glasses.

# References


abstract: Cloud Data centers aim to provide reliable, sustainable and scalable services for all kinds of applications. Resource scheduling is one of keys to cloud services. To model and evaluate different scheduling policies and algorithms, we propose FlexCloud, a flexible and scalable simulator that enables users to simulate the process of initializing cloud data centers, allocating virtual machine requests and providing performance evaluation for various scheduling algorithms. FlexCloud can be run on a single computer with JVM to simulate large scale cloud environments with focus on infrastructure as a service; adopts agile design patterns to assure the flexibility and extensibility; models virtual machine migrations which is lack in the existing tools; provides user-friendly interfaces for customized configurations and replaying. Comparing to existing simulators, FlexCloud has combining features for supporting public cloud providers, load-balance and energy-efficiency scheduling. FlexCloud has advantage in computing time and memory consumption to support large-scale simulations. The detailed design of FlexCloud is introduced and performance evaluation is provided.
author: Minxian Xu, Wenhong Tian, Xinyang Wang, Qin Xiong [^1]
bibliography: CRESS_ComputerJournal3.bib
title: FlexCloud: A Flexible and Extendible Simulator for Performance Evaluation of Virtual Machine Allocation

# Introduction

With various recent advancements in virtualization, like Grid computing, Web computing, utility computing and related technologies, Cloud computing obtains great development. Cloud computing aims to provide both infrastructure and services on demand through the Internet or intranet , and its benefits can be concluded as hiding and abstraction of complexity, virtualized resources and efficient use of distributed resources. Cloud computing allows the sharing, allocation and aggregation of software, computational and storage network resources on demand. Currently, quite a few IT enterprises products, like Amazon EC2 , Google App Engine , IBM blue Cloud and Microsoft Azure have shown their practice of emerging Cloud computing platforms. Whereas there are many challenging issues to be resolved , Cloud computing is still considered in its infancy. Youseff et al. introduce a detailed ontology of dissecting Cloud into five main layers from top to down: Cloud application (SaaS), Cloud software environment (PaaS), Cloud software infrastructure (IaaS), software kernel and hardware (HaaS), and illustrates their interrelations as well as their inter-dependency on preceding technologies. From structure perspective, Cloud data center can be regarded as a distributed network, containing many computing nodes, storage nodes, or network devices. Each node is composite of a series of resources such as CPU, memory, network bandwidth and so on. In this paper, we focus on Infrastructure as a service (IaaS) in Cloud data centers, and proposing general and flexible definition as well as model that could be used by various cloud providers.

An essential technology in Cloud datacenter is resource scheduling. One challenge problem related to scheduling in Cloud data center is to consider allocation and migration of reconfigurable virtual machines and integrated features of hosting physical machines. Different from existing load-balancing scheduling algorithms that consider only physical servers with one factor such as CPU, the new algorithms treat CPU, memory and network bandwidth integrated for both physical machines (PMs) and virtual machines (VMs). Besides that, real-time virtual machine allocation for multiple parallel jobs and physical machines is taken into consideration. With the development of cloud computing, the size and density of the cloud data center become huge, and problems which need to be solved therewith. For instance, how to manage physical resources and virtual resources intensively and use them dynamically, to improve elasticity and flexibility which can improve service and reduce cost and risk management; and how to help customers build flexible, dynamic, and business growth adapting infrastructure as well as ensure the sustainable development in the future.

Because of the uncertainty of network environments, it is extremely hard to research widely for all these problems in real Internet platform. In addition, the network conditions cannot be predicted or controlled accurately, but affect the validation of strategies. A considerate way in research is developing a simulation system, which supports visualized modeling and simulation in large-scale applications in cloud infrastructure. Data center simulation system can describe the application workload statement, which includes user information, data center position, the amount of users and data centers, and the amount of resources in each data center. Using this information, data center simulation system generates response requests and allocates these requests to virtual machines. By using data center scheduling simulation system, researchers can evaluate suitable strategies such as distributing reasonable data center resources, selecting data center to match special requirements, reducing costs, finding efficient scheduling algorithms and so on.

The major contributions of this paper are as following:

-   the proposal of a new cloud simulator, FlexCloud, with light weight design to simulate cloud environment;

-   the design and implementation of a flexible and extendible architecture model that resource, request specification and scheduling algorithms can be easily added;

-   the validation of the simulator, which has been carried out by comparing realistic data with actual results collected from Lawrence Livermore National Lab trace;

-   the performance comparison with CloudSim, which shows FlexCloud has strength in time cost and memory consumption.

The remainder parts of this paper are organized as follows: section 2 describes the related work on virtual machine allocation algorithms and compares existing cloud computing simulators from several categories. At the end of this section, the major contributions of this paper are given. Section 3 demonstrates the architectural model of the newly proposed simulator from several aspects, including its layered architecture, scenario, datacenter modeling, VM requests modeling, scheduling algorithms modeling, implemented performance metrics, VM migration modeling, scheduling process modeling and etc. Section 4 presents implementation details and design patterns adopted in FlexCloud. Section 5 and section 6 demonstrate the validation and evaluation of FlexCloud respectively. Finally, this paper ends with the brief conclusions and a discussion on future work.

# Related works

A mount of research has been conducted in resource scheduling algorithms, which are significant for cloud data centers. Mastroaianni et al. present a self-organizing and adaptive approach for the consolidation of VMs on CPU and RAM resources. Wood et al. introduce techniques for virtual machine migration and propose some migration algorithms. Zhang et al. compare major load balancing scheduling algorithms for traditional web servers. Singh et al. propose a novel load balancing algorithm called VectorDot for handling the hierarchical and multi-dimensional resource constraints by considering both servers and storage in Cloud computing. Doyle et al. propose a system named Stratus to determine the routine decisions for data center requests.

Buyya et al. introduce GridSim toolkit for modeling and simulation of distributed resource management for grid computing. Dumitrescu and Foster introduce GangSim tool for grid scheduling. Buyya et al. introduce modeling and simulations of Cloud computing environments at application level, a few simple scheduling algorithms such as time-shared and space-shared are discussed and compared. CloudSim is one of Cloud computing simulators, which provides: modeling large-scale cloud computing infrastructure; models for the data center, service agency, scheduling and distributing strategies; virtual engines, which is helpful to create and manage several independent and collaborative virtual services in a data center node; switching flexibly between processing cores with space-sharing and time-sharing. CloudAnalyst aims to achieve the optimal scheduling among user groups and data centers based on the current configuration. Both CloudSim and CloudAnalyst are based on SimJava and GridSim . Also CloudSim and CloudAnalyst treat a Cloud data center as a large resource pool and consider only application-level workloads, may not suitable for Infrastructure as a service (IaaS) simulation where each virtual machine as resource is considered to be requested and allocated. A CloudSim-based simulation tool considering DVFS energy model is proposed in . Kliazovich et al. propose an energy-aware simulation environment named GreenCloud for Cloud datacenters . Nunez et al. introduce a new simulator of cloud infrastructure named iCanCloud using C++ and compare the performance with CloudSim.

```latex
\begin{table*}
\scriptsize
\caption{Summary of Cloud Simulators}
\begin{center}
\begin{tabular}{l|l|l|l|l|l}
\hline Items & CloudSim & MDCSim& GreenCloud & iCanCloud & FlexCloud
\\\hline
\hline Platform & any & CSIM & NS2 & OMNET, MPI & any \\
\hline Programming Language & Java & C++/Java &C++/OTcl & C++ & Java \\
\hline Availability  &Open Source & Commercial&Open Source&Open Source & Open Source \\
\hline Graphical Support & Limited (Via CloudAnalyst) &None & Limited (Via Nam)&Full&Full\\
\hline Physical Models& None &None &Limited (Via Plug-in)&Full&Full\\
\hline Models for public cloud & None &None&None&Amazon&Amazon\\
\hline Support for Parallel experiments & No &No &No&Yes&No \\
\hline Support for Energy Consumption Model&Yes&Yes&Yes&No&Yes\\
\hline Support for Migration algorithms &Yes&No&No&No&Yes\\
\hline
\end{tabular} \\
\end{center}
\end{table*}
```

Table 1 shows the comparison of some state-of-art cloud simulators as well as FlexCloud proposed in this paper. We compare these cloud simulators from several categories.

Platform: CloudSim and FlexCloud are both implemented with Java, so they can be executed on any machine installed JVM. Built in GridSim and SimJava, CloudSim is heavy to execute. MDCSim is written in CSIM, as for GreenCloud and iCanCloud, they are based on NS2 and OMNET respectively.

Language: The languages implemented with the simulators are related to the platforms. CloudSim and FlexCloud are implemented with Java, MDCSim can be implemented with C++ and Java, and GreenCloud needs combining C++ and OTcl, which is difficult for developers.

Availability: Only MDCSim is commercial, and other four simulators are free or open-source. FlexCloud can be fetched from .

Graphical support: MDCSim doesn¡¯t support interface operations. The original CloudSim support no graphical interface, but with CloudAnalyst, the graphical interface are supported. However, full support is not provided in CloudAnalyst, in a whole scheduling process, only the configurations and results can be presented. So we label it ¡°limited¡±, the same reason for GreenCloud. FlexCloud and iCanCloud support whole scheduling process to be showed on the interfaces.

Physical models: iCanCloud and FlexCloud provide detailed simulation for physical analogs for the scheduling. GreenCloud needs to use a plug-in to simulate that.

Models for public cloud providers: Both iCanCloud and FlexCloud use the model suggested by Amazon, in which physical machine and virtual machine specifications are pre-defined.

Parallel experiments: Supporting for multiple machines running the experiments together is a main feature of iCanCloud and that feature is under development. As for FlexCloud, we are working to implement that function as well.

Power consumption model: Except for iCanCloud, other four simulators can support power consumption modeling.

Migration algorithm: CloudSim and FlexCloud support migration algorithms, while other 3 simulators haven't supported that.

In our teaching practice in our university, we have adopted CloudSim, a mature simulator, as a teaching tool assisted, but according to the students¡¯ feedback, CloudSim is a bit complex to use and heavy to execute. That complexity is also a feature of iCanCloud. As for MDCSim, a commercial tool, is not appropriate for researching. Apart from that, it¡¯s not easy to use several languages together in GreenCloud, since it is implemented with C++ and OTcl.

The main contribution of FlexCloud lies in that it is implemented with light weight design, flexible to extend as well as easy to start. Besides the benefits for teaching, we also cooperate with a company researching in resource scheduling to boost the functions of FlexCloud under multi-datacenter environment. They would use FlexCloud to explore suitable algorithms for their company applications.

# The Architectural Model of FlexCloud

Fig.1 shows the overview architecture of FlexCloud with layered components. The top layer is Client Layer that provides the interface for user to configure requests properties and have results feedbacks from lower layers. At this layer, a GUI implemented with Java Swing supports user to configure algorithm types, set PM and VM specifications and select scheduling algorithms. After all settings are completed, the defined configurations would be submitted to lower layer and a sequence of scheduling steps would be processed. Comparison diagrams as well as result outputs would be sent as feedback to Client Layer.

At lower layer, a Requests Broker is implemented at Broker Layer acting as a mediator between Client Layer and Scheduler Layer. This Layer is responsible for verifying the inputs from Client Layer and transforming the settings into recognized commands at Scheduler Layer. For instance, the number of VM requests submitted from Client Layer would be written into a configuration file, which could be read in the process of scheduling at Scheduler Layer. Scheduler Layer implements the core functions for FlexCloud system. At this layer, the scheduling process is defined: VM Requests Generation component generates the VM requests with configured properties on user interface; Datacenter Scheduler component schedules the particular algorithms to allocate VMs to corresponding PM according to algorithms; VM Requests Allocation component manages the allocated VMs, including checking the allocation conditions and removing VMs at the end of their lifecycles. At bottom layer, Resource Layer contains a Resource Management component providing resource that VM requests require and supporting services for higher levels. Besides the component, the physical resource, such as servers, network and storage are resources of the whole system.

Fig.2 shows an application scenario with FlexCloud. This figure shows the three main components: user, FlexCloud scheduler center and other computing centers. The FlexCloud scheduler center is responsible for the following main tasks: (1) accepting the VM requests sent by users; (2) managing computing centers that in service; (3) finding available computing unit to allocate requests; (4) sending feedback information to users. Computing centers represent a pool of Physical Machines (PMs) or Virtual Machines (VMs), each one configured with a pre-defined specification such as CPU, memory and storage. Users are represented as component that submits a set of jobs to be allocated to specific PM in computing center. These submissions submitted directly to the FlexCloud scheduler center. Then, the requests are managed by this module to be allocated to specific PM in the corresponding data center. After all requests have been processed, a feedback report would be sent back to the user.

## Modeling the datacenter in FlexCloud

From computing resource point of view, a data center consists of a number of physical servers (PMs), network devices, storages and other related equipment. A PM contains several kinds of resource, like CPU, memory, storage and bandwidth, etc. Before VM requests are coming, the PMs are at the state of turned-on, which means the class of Physical Machine is instantiated in FlexCloud. The number of instances depends on the number of PMs would provide services.

| PM Pool Type | Compute Units | Memory | Storage |
|:-------------|:--------------|:-------|:--------|
| Type 1       | 16 units      | 30GB   | 3380GB  |
| Type 2       | 52 units      | 136GB  | 3380GB  |
| Type 3       | 40 units      | 14GB   | 3380GB  |

3 types of physical machines (PMs) suggested

In TABLE 2, the 3 suggested types of heterogeneous PMs in FlexCloud are listed, and the configuration can be dynamically set. The type and property values, like CPU, memory, storage and power, are recorded in a configuration XML file (in Fig. 3), which would be loaded into system. Because these property values are in XML file, modification can be easily done either for exactly value or new added property elements. For instance, if more types of PMs are needed, the pair $<pminfo>type-id<pminfo>$ could be created and other values of this type PM could so also be added. Besides the load balance algorithms, we also implement energy-saving algorithms that contain a new property named power consumption. This property is added in the configuration XML file and corresponding methods are added in class PhysicalMachine. The corresponding methods in class PhysicalMachine are responsible for accessing these property values.

More detailed information related to comparison indices can be found in Section 3.3. In datacenter model, the left resource capacity decides whether a VM request can be allocated to that PM. At initialization stage, PM has a full capacity resource to offer services. Either the allocation or remove operation would update the available capacity value and influence the later requests allocation.

## Modeling VM requests in FlexCloud

We use a simple example to show how VM requests are modeled in FlexCloud in Fig. 4. Slots $\#1, \#2, \ldots, \#6$ represent the time slots in discrete time, which can be treated as a second or a minute that a request is in. For instance, VM2 occupies time slot 3 to 5, so lifecycle of VM2 is 3 slots. The value 0.0625 is proportion of resource occupation, meaning that VM2 would occupy 6.25% resource of the PM it is allocated to, during time slot 3 to 5. In our model, several VM requests can share the capacity of the same PM at the same time slot only if the capacity is enough.

TABLE 3 shows the corresponding CPU, memory, storage values for different VMs. Also for extensible reason, these property values are also recorded into a configuration XML file. Once a VM request is allocated to a PM, the left resource capacity would be decreased by the value of that request, and the capacity is increased back when request is released.

In FlexCloud, several VM requests generation approaches have been implemented, in which requests can be generated in Poisson, Normal and Random distributions. When the specific distribution is selected, the start time or duration of the generated requests would follow the distribution. In section 5, we would show the data collected from different distributions. Moreover, it¡¯s available for FlexCloud to import requests data from file in the *Generate VMs* step (see section 3.6), which means it can be tested under realistic data.

| Compute Units | Memory | Storage | VM Type |
|:--------------|:-------|:--------|:--------|
| 1 units       | 1.7GB  | 160GB   | 1-1(1)  |
| 4 units       | 7.5GB  | 850GB   | 1-2(2)  |
| 8 units       | 15GB   | 1690GB  | 1-3(3)  |
| 6.5 units     | 17.1GB | 420GB   | 2-1(4)  |
| 13 units      | 34.2GB | 850GB   | 2-2(5)  |
| 26 units      | 68.4GB | 1690GB  | 2-3(6)  |
| 5 units       | 1.7GB  | 350GB   | 3-1(7)  |
| 20 units      | 7GB    | 1690GB  | 3-2(8)  |

8 types of virtual machines (VMs) in Amazon EC2

## Modeling Scheduling Algorithms in FlexCloud

Four kinds of scheduling algorithms are provided in FlexCloud based on scheduling goals and request types. For request types, scheduling algorithms can be divided into online algorithms and offline algorithms, the difference lies in whether the requests information is all known before scheduling. Requests would come and be operated one by one in online algorithm, while requests sequence can be adjusted by processing time or end time because all requests information have been collected before scheduling in offline algorithms. Another division principle is via goal: we consider load balancing and energy saving in FlexCloud.

When comparing the effects of different algorithms, the scheduling process would be same except that the scheduling algorithms are different. For online load balancing comparison, Random, Round-Robin (Round), List Scheduling (LS) algorithms have been implemented. Under the layered architectural model and related design pattern (introduced in later section), new created algorithms can be added to scheduling algorithm library, without influencing other existed algorithms.

We take the Random algorithm, one of the simplest algorithms, as an example to show scheduling algorithm could be modeled, mapped and extended in FlexCloud. Fig.5 shows the pseudo-code of Random algorithm. After a data center scheduler has initialized the PMs and VM requests, Random algorithm would randomly generate an index in the range of 0 and $M$-1 (line 3) for PMs. Then VM request will be allocated to the PM with generated index (line 5), if allocation is successful by checking whether the PM has available resource, then allocated PM needs updating its left capacity (line 7), another index would be generated if allocation is failed and the VM request should be allocated again with a new index (line 8-9). As VM requests have lifecycles, the VM requests should be released from hosting PM to prepare for other VM requests' allocation (line 11 to 12) after their end-time expired. This example is based on singe data center, while under multiple data centers, the index generation process would be involved with data center id generation, rack id generation and PM id generation rather than only PM id generation.

The algorithm process of R-R and LS is quite similar to Random except the way the index is generated for PMs or VMs. In R-R algorithm, index generation is in a round robin way while the index refers to the PM with the least average utilization in LS algorithm. The more complex algorithms, Post Migration algorithms and Prepartition Algorithm, that considering migrations operations introduced in section 3.5 could also be modeled based on these principles. As for offline load balancing algorithms, a procedure of request processing should be added before line 4. The processing procedure may change the requests order by processing time or end time or other features. Also, the process of online/offline energy-saving algorithms taken is similar to online/offline load balancing algorithms.

## Performance Metrics in FlexCloud

In this section, we introduce the major performance metrics we used in FlexCloud:

For load balancing algorithms:
Average utilization: Each PM would have the utilization value in scheduling process, and average utilization is the arithmetic average value of all PMs in the data center;
PM resource: $PM_i(i,PCPU_i,PMem_i,PStorage_i)$, $i$ is the index number of PM, $PCPU_i,PMem_i,PStorage_i$ are the CPU, memory, storage capacity of that a PM can provide.
VM resource:
$VM_j(j,VCPU_j,VMem_j,VStorage_j,T_j^{start},T_j^{end})$, $j$ is the VM type ID, $VCPU_j,VMem_j,VStorage_j$ are the CPU, memory, storage requirements of $VM_j$, $T_j^{start},T_j^{end}$ are the start time and end time, which are used to represent the life cycle of a VM.
Time slot: we consider a time span from 0 to T be divided into parts with same length. Then $n$ parts can be defined as $[(t_1-t_0),(t_2-t_1),\ldots,(t_n-t_{n-1})]$, each time slot $T_k$ means the time span $(t_k-t_{k-1})$.
Average CPU utilization of $PM_i$ during slot 0 and $T_n$: $$PCPU_i^U=\frac{\sum_{k=0}^{n} (PCPU_i^{T_k}\times T_k)}{\sum_{k=0}^{n} T_k}$$ And memory $PMem_i^U$ and storage $PStorage_i^U$ utilization of both PMs and VMs can be computed in the same way. Similarly, average CPU utilization of a VM can be computed.
Integrated load imbalance value $ILB_i$ of $PM_i$. The variance is widely used as a measure of how far a set of values is spread out from each other in statistics. Using variance, an integrated load imbalancing value $ILB_i$ of server $i$ is defined $$\begin{aligned}
ILB_i =& \frac{(Avg_i-CPU_u^A)^2}{3} +\frac{(Avg_i-Mem_u^A)^2}{3} \nonumber \\
&+\frac{(Avg_i-Storage_u^A)^2}{3}
\end{aligned}$$ where $$Avg_i = \frac{PCPU_i^U+PMem_i^U+PStoarge_i^U}{3}$$ and $CPU_u^A,Mem_u^A,Storage_u^A$ are respectively the average utilization of CPU, memory and storage in a Cloud data center.
$ILB_i$ is applied to indicate load imbalance level comparing utilization of CPU, memory and network bandwidth of a single server itself.
Makespan: is as same as traditional definition, and therefore the capacity$\_$makespan of all PMs can be formulated as below: $$capacity\_makespan = \max_i{(L_i)}$$ Load efficiency (skew of makespan): is defined as the (minimal average load divided by maximal average load) on all machines: $$skew(makespan) = \frac{\min_i(L_i)}{\max_i(L_i)}$$ where $L_i$ is the load of PM $i$. Skew shows the load balancing efficiency to some degree.
Capacity$\_$makespan: In any allocation of VM requests to PMs, we can let $A(i)$ denote the set of VM requests allocated to machine $PM_i$, under this allocation, machine $PM_i$ will have total loads, $$L_i=\sum_{j\in A(i)}c_jt_j$$ Based on the above definitions and equations, we have developed another metric, capacity$\_$skew on load balancing algorithm for the new situation as follow:
Skew of capacity$\_$makespan is defined as the minimal capacity$\_$makespan over maximal capacity$\_$makespan on all machines (referring to equation (6)): $$skew(capacity\_makespan) = \frac{\min{\sum_{j\in A(i)}c_jt_j}}{\max {\sum_{j\in A(i)}c_jt_j}}$$ where $c_j$ is the capacity (for example CPU) requests of $VM_j$ and $t_j$ is the span of request $j$ (i.e., the length of processing time of request $j$).

For energy saving algorithms, following indices are provided:
1) The total number of PMs turned-on during the scheduling;
2) Rejected number of VM requests: VM requests which cannot be served by the data center resources;
3) Total energy consumption: the energy consumption of all PMs (including VMs allocated on them); a less total energy consumption value reflects a better energy saving effect for a given set of requests.

In , authors found that CPU utilization is typically proportional to the overall system load, and proposed a power model defined in equation (8): $$P(u) = kP_{max} + (1-k)P_{max}u$$ where $P_{max}$ is the maximum power consumed when the server is fully utilized; $k$ is the fraction of power consumed by the idle server (studies show that on average it is about 70%); and $u$ is the CPU utilization. This paper focuses on CPU power consumption, which accounts for main part of energy comparing to the other resources such as memory, disk storage and network devices. In FlexCloud, we use the power model defined in (8). Equation (8) is further reduced to (9): $$P = P_{min} + (P_{max} - P_{min})u$$ where $P_{min}$ is the power of given PM when its CPU utilization is zero (the PM is idle without any VM running). In real environment, the utilization of the CPU may change over time due to the workload variability. Thus, the CPU utilization is a function of time and is represented as $u(t)$. Therefore, the total energy consumption by a PM ($E_i$) can be defined as an integral value of the power consumption function over a period of time as in (10): $$E_{i} = \int_{t0}^{t1} P(u(t))dt$$ If $u(t)$ is constant over time, for example average utilization is adopted, $u(t) = u$, then $E_i = P(u)\times(t_1 - t_0)$.

The total energy consumption of a cloud data center is computed as (11): $$E_{DC} = \sum_{i=1}^{n}E_i$$ It is the sum of energy consumed by all PMs. Notes that energy consumption of all VMs on PMs is included.

Also confidence intervals can be calculated for different metrics as follows: Let $x_1, x_2, x_3,..., x_n$ be the calculated metrics (such as $IBL_{tot}$  and  $E_{cdc}$ values etc.) from $n$ times of repeated simulations. Then the mean is $$x_{mean}=\frac{1}{n}\sum_{i=1}^{n} x_i$$ and the standard deviation $s$ is $$s=\sqrt{\frac{\sum_{i=1}^{n} (x_{mean}-x_i)^2}{n-1}}$$ and the confidence interval at 95$\%$ confidence is given by $$(x_{mean}-1.96\frac{s}{\sqrt{n}}, x_{mean}+1.96\frac{s}{\sqrt{n}})$$
Above are basic the metrics that already implemented in FlexCloud. Other metrics could also be included for further research.

## Modeling Virtual Machine Migrations in FlexCloud

There is lack of virtual machine migration modeling in existing simulation tools. In , the detailed algorithms about migration are introduced and compared. In this section, we provide brief introduction to virtual machine migration modeling in FlexCloud. The key difference from allocation is that the migration objectives and the choose of resource and destination PMs. Two typical migration algorithms are introduced in FlexCloud:

Post Migration algorithm: Firstly, it processes the requests in the same way as LPT (Longest Processing Time first) does. Then the average capacity$\_$makespan of all jobs is calculated. The up-threshold and low-threshold of the capacity$\_$makespan for the post migration are calculated through the average capacity$\_$makespan multiplied by a factor (in this paper we set the factor as 0.1, so the up-threshold is average capacity$\_$makespan multiplied by 1.1 and the low-threshold is multiplied by 0.9). Off course the factor can be set dynamically to meet different requirements; however, the larger the factor is, the higher imbalance is. A migration list is formed by collecting the VMs taken from PMs with capacity$\_$makespan higher than the low-threshold. The VMs would be taken from a PM only if the operation would not lead the capacity$\_$makespan of the PM to be less than the low threshold. After that, the VMs in the migration list would be re-allocated to a PM with capacity$\_$makespan less than the up-threshold. The VMs would be allocated to a new PM only if the operation would not lead the capacity$\_$makespan of the PM to be higher than the up-threshold. There may be still some VMs left in the list, finally the algorithm allocates the left VMs to the PMs with the lowest capacity$\_$makespan until the list is empty.

Capacity_makespan Prepartition Algorithm: novel work proposed by ourselves. For a given set of VM reservations, let us consider there are $m$ PMs in a data center and denote OPT as the optimal solution for a given set of $J$ VM reservations. Firstly define $$P_0=max\{max_{j=1}^{J} {CM_j},\frac{1}{m}\sum_{j=1}^{J} CM_j\}\leq OPT$$ $P_0$ is a lower bound on OPT. The Capacity_makespan Prepartition algorithm is introduced in detailed in . It firstly computes balance value by equation (15), defines partition value ($k$) and finds the length of each partition (i.e. $\lceil P_0/k\rceil$, which is the max time length a VM can continuously run on a PM). For each request, Prepartition equally partitions it into multiple $\lceil P_0/k\rceil$ subintervals if its CM is larger than $\lceil P_0/k\rceil$, and then finds a PM with the lowest average capacity$\_$makespan and available capacity, and updates the load on each PM. After all requests are allocated, the algorithm computes the capacity$\_$makespan of each PM and finds total partition (migration) numbers. For practice, the scheduler has to record all possible subintervals and their hosting PMs of each request so that migrations of VMs can be conducted in advance to reduce overheads.
FlexCloud therefore can evaluate the performance of different migration algorithms; the evaluation process is similar to allocation algorithms.

## The Scheduling Process in FlexCloud

The major steps of the scheduling process in FlexCloud are as followings:
1). Booting PMs: it loads the configuration XML file containing PM specifications set by user from user interface. After needed information is collected, instances of PMs are created to prepare for VM allocation.
2). Generating traces (VM requests): it loads the configuration XML file containing VM specifications and VM traces from user interfaces.
3). Comparing scheduling algorithms: two or more iterators would collect the compared algorithms and compared indices. All selected algorithms will be compared and corresponding indices are collected.
4) Output results: the comparison results are outputed in both text or diagrams format.

For building a more flexible system, the scheduling process only defines the basic framework process and customization may be improved based on this process. Before booting PMs, the PM specifications can be modified in configuration file. As for generating VM requests traces, besides the configuration file, more VM requests creation methods can be implemented. Various algorithms can be developed in algorithm scheduling process and other results format may be adopted for better visual effects. Fig. 6 shows a scheduling process combing user interface configurations and basic scheduling process.

#  Implementation of FlexCloud

In this section, we will introduce the detailed implementation of FlexCloud from design patterns' point of view. The design principles are mainly aiming at satisfying agile system goals with flexibility and extendibility.

## Main Features in FlexCloud

Considering design principles, FlexCloud mainly has following novel features:
(1) FlexCloud is built on Java platform and can be run on a single computer installed JVM to simulate large scale cloud infrastructure as a service (IaaS). A computer with 4 GB memory can simulate larger scale applications. We test the condition when the Java environment would throw an OutOfMemory exception by increasing requests gradually. With a 4 GB memory computer, experiments can simulate scheduling process of more than 100,000 requests. We have extended our tests with computers with 2GB memory, that configuration can simulate requests ranging from 25,000 to 50,000.
(2) A user-friendly GUI is provided and lots of customized configurations can be set to satisfy various simulation assumptions. The basic operation includes: select algorithm type, set VM numbers, set average duration, set start time, set total number of PMs, select comparison algorithms and indices. Of course, without GUI, user can also simulate a cloud datacenter and scheduling process in .java class file as well.
(3) A scheduling process framework is defined, each step of process can be extended easily in agile style.
(4) New scheduling algorithms and performance metrics are flexible and extendable to add in; currently load-balancing and energy-efficiency scheduling algorithms are considered.
(5) Virtual machine migration is modeled, this is still lack in current simulation tools.

## Design Patterns in FlexCloud

FlexCloud has adopted some design patterns to meet the extendible goal. Fig.7 shows decorator pattern to meet the requirement that requests generation approaches may differ. Class CreateVMDecoratorA and CreateVMDecorateB extend the createVM() method of CreateVM and add a new behavior method. With this pattern, when new requests generation approaches are needed, we can rewrite the method addedBehavior(). Under this method, function of class CreateVM can be dynamically added or deleted. For instance, new resource is needed, resource collection codes can be put in the addedBehavior() method rather than change the existing codes or add new classes.

Fig.8 shows the Abstract Factory pattern to meet the requirement of different compositions of requests generation approaches and scheduling algorithms. An instance of LoadBalanceFactory would combine a request generation approach from CreateVM and scheduling algorithms from generalization of AllocateAlgorihm. These different combinations can produce diverse scheduling process, like set requests order by processing time and scheduled by a subtype of class OnlineAlgorithm. Adopting this design pattern can avoid fixed composition and gain a better extendable effect.

Strategy pattern in Fig.9 defines a series encapsulate scheduling algorithm classes: Random, Round-Robin, and OLRSA \[18\] for OnlineAlgorithm and LPT (Longest Process Time) etc. for OfflineAlgorithm, which can be substituted with each other, enabling scheduling algorithms be independent on the changes from users. With strategy design pattern, when a new algorithm is joined, only allocate() method in new joined algorithm should be implemented. After that, the new joined algorithm can work as same as existing algorithms.

Fig.10 shows the Iterator pattern to meet the requirement of algorithm and indices results comparison in data centers. After user have selected the comparison indices and algorithms on user interface, the selected algorithms and indices would be added to separate list, at the same time, Iterator for algorithm and index, Index Iterator and Algorithm Iterator, would be generated. When outputting results, the Iterator would schedule algorithms in Iterator one by one and output indices results with showIndex() method in order. To satisfy the Iterator, both algorithms and indices should extend from their base class. The strength for this design pattern is also a favorable extendibility. It's intelligent when new algorithms and indices are implemented, the Iterator would compare results only if they are appended to it.

## Communications Between Entities

Fig.11 depicts the flow of communication among important FlexCloud entities. At the beginning of the simulation, a DataCenter entity sends necessary messages that BootPM entity needs to start PMs providing services in datacenter. CreateVM entity would also accept messages it needs to create VM requests. AlgorithmIterator and IndexIterator entities act to run scheduling algorithms and send calculated indices values back to DataCenter entity.

The communication flow described above is a basic flow in a simulated experiment. Some variations in this flow are possible depending on the scheduling process. For example, before bootPM() and createVM(), the message sent by a datacenter should be verified.

# Validation of FlexCloud

To validate the accuracy of FlexCloud, we have designed some test cases to compare the theoretical results and simulation results. In this section, we use LS (List Scheduling), LPT(Longest Processing Time First), EDF(End-time Decreasing First) algorithms to compare theoretical and simulation results.

| LS Indices              | Theoretical | Simulation |
|:------------------------|:------------|:-----------|
| Average Utilization     | 0.5         | 0.5        |
| Imbalance Degree        | 0.0         | 0.0        |
| Makespan                | 0.5         | 0.5        |
| Skew(makespan)          | 1           | 1          |
| Capacity_makespan       | 50          | 50         |
| Skew(capacity_makespan) | 1           | 1          |

Theoretical and simulation results comparison of LS algorithm

The test cases we designed are easily theoretically calculated and can reflect some general situations. For LS algorithm that always allocate a VM to the PM with the lowest load, we set that there are 100 PMs and 100 VMs requests both in the same types, the start-time of requests are ordered in increasing sequence, $1,2,3,\ldots, 100$, and all requests duration are 100 and require capacity is 0.5 of a PM. Since PMs number and VMs number are same in this case, LS algorithm works as Round-Robin algorithm, that means each PM would undertake a VM task. Then we calculate the values in theoretical way and simulation, same results have been observed and shown in TABLE 4.

| LPT Indices             | Theoretical | Simulation |
|:------------------------|:------------|:-----------|
| Average Utilization     | 0.505       | 0.505      |
| Imbalance Degree        | 0.0         | 0.0        |
| Makespan                | 1           | 1          |
| Skew(makespan)          | 1           | 1          |
| Capacity_makespan       | 50.5        | 50.5       |
| Skew(capacity_makespan) | 1           | 1          |

theoreticcal and simulation results comparison of LPT algorithm

We also design a test case for LPT algorithm, an offline algorithm that VM requests can be reordered by processing time before they are allocated. In this case, there are 50 PMs and 100 VMs both in the same types, each request requires 0.5 capacity of a PM and starts at $1,2,3,\ldots,100$, and the durations of VMs are ordered in decrease order from 100 to 1 as $100,99,98,\ldots,1$. Same results have been observed and collected in TABLE 5.

| EDF Indices       | Theoretical | Simulation |
|:------------------|:------------|:-----------|
| Power Consumption | 250000      | 250000     |
| Rejected Number   | 10          | 10         |
| Turned on PMs     | 20          | 20         |

theoretical and simulation results comparison of EDF algorithm

For energy saving algorithm EDF, it should be noticed that comparison indices are different and requests are ordered by end-time. In this case, we set that there are 20 PMs and 50 VMs both in the same types, the start-times of VM requests are ordered in increasing as $1,2,3,\ldots,50$ and end-times are decreasing as $100,99,98,\ldots, 51$. Each VM requires 0.5 capacity of a PM. We adopt the energy saving model referred to section 3.4 and assume $P_{min} = 300$, $P_{max} = 500$. Same theoretical and simulation values have been collected in TABLE 6. Referring to the collected data in TABLE 5 and 6, the results show the correctness of FlexCloud.

# Evaluations

In this section, we provide more performance evaluations for FlexCloud, including evaluations for different algorithms with basic and advanced settings.

## Basic Algorithm Performance Evaluations

To begin with, we compare scheduling algorithms performance with basic settings, and show the comparison diagrams generated by FlexCloud. The related settings are as followings:
1) Algorithm type: is online load balancing;
2) PM specifications: using suggested specifications in Amazon EC2 shown in Table 2. PM type1 number is 50, type2 and type3 number are set as 0 to simplify simulation;
3) VM requests: using suggested specifications in Amazon EC2 as shown in Table 3 and 3, and requests are generated under Normal Distribution.
4) Algorithms for comparison: Random, RoundRobin (R-R) and List Scheduling algorithm (LS, referring to section 3.3);
5) Indices for comparison: average utilization, imbalance degree, capacity_makespan, skew of makespan, skew of capacity_makespan.
FlexCloud provides several output formats for further analysis, like diagram outputs, text outputs or outputs in Excel file. The diagram output results are showed as bar chart presented in Fig. 12, which is composite of three small diagrams. As seen from these diagrams, it can be concluded that LS overwhelms the other two algorithms on imbalance degree, makespan, skew of makespan, and the skew of capacity_makespan with the settings. It¡¯s easy to understand as LS algorithm dynamically allocates VM requests based on the PM loads while Random and RoundRobin algorithms do not collect real-time load information from PMs.

## Advanced Algorithm Performance Evaluations

To extend performance evaluations, we also compare scheduling algorithms performance with advanced settings and collect the comparison data. The related settings are as following:
1) Algorithm type is offline load balancing;
2) PM specifications: using suggested specifications in Amazon EC2 shown in Table I. PMs with different numbers are considered. PMs numbers are varying from 15, 30, 60 to 240 and each type of PMs occupies about 1/3 of total PMs numbers;
3) VM requests: using suggested specifications in Amazon EC2 shown in Table II. We adopt the log data at Lawrence Livermore National Lab (LLNL) to reflect realistic data generation. The log contains months of records collected by a large Linux cluster and has characteristics consistent with our problem model. Each line of data in that log file includes 18 elements, while we only need the request-ID, start-time, duration and number of processors (capacity demands) in our simulation. We convert the units from seconds in LLNL log file into minutes, as we design 5 minutes to be a time slot length;
4) Algorithms for comparison: RoundRobin (R-R), Longest Processing Time first (LPT, referring to section 5), Post Migration Algorithm (MIG, referring to section 3.5), Capacity_makespan Prepartition Algorithm (CMP, referring to section 3.5);
5) Indices for comparison: average utilization, imbalance degree, longest process time and capacity_makespan.

Fig.13 to Fig.14 show the average utilization, imbalance degree, makespan and capacity$\_$makespan comparison for different algorithms with LLNL data trace. From these figures, we can notice that CMP algorithm has better performance than other algorithms in average utilization, imbalance degree, makespan, capacity$\_$makespan. CMP algorithm has 10$\%$-20$\%$ higher average utilization than MIG and LPT, and 40$\%$-50$\%$ higher average utilization than Random-Robin (R-R). Prepartition algorithm has 10$\%$-20$\%$ lower average makespan and capacity$\_$makespan than MIG and LPT, and 40$\%$-50$\%$ lower average makespan and capacity$\_$makespan than R-R.

Besides the above evaluations, we also vary the partition number $k$ from 4, 8 to 10 to compare the load balance affects. Fig.15 presents imbalance degree of Capacity_makespan Prepartition algorithm with different $k$ values. It's easy to understand that a larger $k$ value would produce a better load balance, which would lead to more partitions, and more partitions could achieve better load balance effects. It can be observed that whatever numbers of migrations to taken, Post Migration algorithm (MIG) just cannot achieve the same level of average utilization, makespan and capacity$\_$makespan as Capacity_makespan Prepartition does.

# Conclusions and Further work

In this paper, we introduce the FlexCloud, a novel simulator for performance evaluation of virtual machine allocation in Cloud data centers. It is flexible, scalable to simulate resource scheduling in cloud data centers. A complete simulation framework has been built and introduced.

There are a few research directions for extending the simulator:

-   Considering more scheduling algorithms. In FlexCloud, we already implemented load-balancing and energy-efficiency, other scheduling algorithms such as cost-oriented or reliability-oriented algorithms can be added in easily.

-   Evaluate performance by datasets from real traces. Currently we are collecting data from real cloud applications, more evaluating results can be provided by real traces and benchmarks.

-   Providing more visual outputs such as dashboards and logical view of different data centers and their resource usages. This information is very important for managers and operators to have.

-   Considering more infrastructures, such as networking devices. Currently FlexCloud considers bandwidth requests and allocations. The network devices such as three-tire switches and routers distributed in different data centers are under modeling consideration.

    # Acknowledgment

    This research is partly sponsored by the National Natural Science Foundation of China (NSFC) (Grant Number:61150110486), and by Scientific Funding of Central University (2012-2014).

[^1]: Dr. Tian, Mr. Wang and Miss Xiong are in the school of Computer Science, University of Electronic Science and Technology of China (UESTC), 610054; Mr. Xu is in the school of software engineering of UESTC.