Modern web services rely on Content Delivery Networks (CDNs) to efficiently deliver contents to end users. In order to minimize the experienced communication cost, it is necessary to send the end user's requests to the nearest servers. However, it is shown that this naive method causes some servers to get overloaded. Similarly, when distributing the requests to avoid overloading, the communication cost increases. This is a well-known trade-off between communication cost and load balancing in CDNs. In this work, by introducing a new meta-heuristic algorithm, we try to optimize this trade-off, that is, to have less-loaded servers at lower experienced communication cost. This trade-off is even better managed when we optimize the way servers update their information of each others' load. The proposed scheme, which is based on Honeybee algorithm, is an implementation of bees algorithm which is known for solving continuous optimization problems. Our proposed version for CDNs is a combination of a request redirecting method and a server information update algorithm. To evaluate the suggested method in a large-scale network, we leveraged our newly developed CDN simulator which takes into account all the important network parameters in the scope of our problem. The simulation results show that our proposed scheme achieves a better trade-off between the communication cost and load balancing in CDNs, compared to previously proposed schemes.
@cite_24: Redundancy is an important strategy for reducing response time in multi-server distributed queueing systems. This strategy has been used in a variety of settings, but only recently have researchers begun analytical studies. The idea behind redundancy is that customers can greatly reduce response time by waiting in multiple queues at the same time, thereby experiencing the minimum time across queues. Redundancy has been shown to produce significant response time improvements in applications ranging from organ transplant waitlists to Google’s BigTable service. However, despite the growing body of theoretical and empirical work on the benefits of redundancy, there is little work addressing the questions of how many copies one needs to make to achieve a response time benefit, and the magnitude of the potential gains. In this paper we propose a theoretical model and dispatching policy to evaluate these questions. Our system consists of k servers, each with its own queue. We introduce the Redundancy-d policy, u...
@cite_10: Load balancing for distributed servers is a common issue in many applications and has been extensively studied. Several distributed load balancing schemes have been proposed that proactively route individual requests to appropriate servers to best balance the load and shorten request response time. These schemes do not require a centralized load balancer. Instead, each server is responsible for determining, for each request it receives from a client, to which server in the pool the request should be forwarded for processing. We propose a new request routing scheme that is more scalable to increasing number of servers and request load than the existing schemes. The method combines random server selection and next-neighbor load sharing techniques that together prevent the staleness of load information from building up when the number of servers increases. Our simulation shows that it outperforms existing schemes under a piggyback-based load update model.
@cite_23: We consider the following natural model: customers arrive as a Poisson stream of rate spl lambda n, spl lambda <1, at a collection of n servers. Each customer chooses some constant d servers independently and uniformly at random from the n servers and waits for service at the one with the fewest customers. Customers are served according to the first-in first-out (FIFO) protocol and the service time for a customer is exponentially distributed with mean 1. We call this problem the supermarket model. We wish to know how the system behaves and in particular we are interested in the effect that the parameter d has on the expected time a customer spends in the system in equilibrium. Our approach uses a limiting, deterministic model representing the behavior as n spl rarr spl infin to approximate the behavior of finite systems. The analysis of the deterministic model is interesting in its own right. Along with a theoretical justification of this approach, we provide simulations that demonstrate that the method accurately predicts system behavior, even for relatively small systems. Our analysis provides surprising implications. Having d=2 choices leads to exponential improvements in the expected time a customer spends in the system over d=1, whereas having d=3 choices is only a constant factor better than d=2. We discuss the possible implications for system design.
The power of multiple choices @cite_23 is a classic paradigm for load balancing which is used in many practical implementations of request routing algorithms. Simplicity and achieving much better results in comparison to a simple randomized method are the main advantages of this paradigm. In @cite_10 the authors have implemented an extension to this paradigm, called . In this method, each server can receive the clients' requests and forward them to the most appropriate server. The best server is chosen between a random server and it's next neighbor. The next-neighbor load sharing method leads to better response time as the authors claimed. In addition, Gardner and Balter have used the power of two choices concept to propose a new theoretical dispatching model @cite_24 . In this algorithm, each request is copied in multiple servers' queues. This way, the requests would wait in multiple queues at the same time, resulting in the minimum waiting time across queues. The authors showed that copying each request in only two servers will reduce average response time noticeably.