WebRTC enables browsers to exchange data directly but the number of possible concurrent connections to a single source is limited. We overcome the limitation by organizing participants in a fat-tree overlay: when the maximum number of connections of a tree node is reached, the new participants connect to the node's children. Our design quickly scales when a large number of participants join in a short amount of time, by relying on a novel scheme that only requires local information to route connection messages: the destination is derived from the hash value of the combined identifiers of the message's source and of the node that is holding the message. The scheme provides deterministic routing of a sequence of connection messages from a single source and probabilistic balancing of newer connections among the leaves. We show that this design puts at least 83 of nodes at the same depth as a deterministic algorithm, can connect a thousand browser windows in 21-55 seconds in a local network, and can be deployed for volunteer computing to tap into 320 cores in less than 30 seconds on a local network to increase the total throughput on the Collatz application by two orders of magnitude compared to a single core.
@cite_29: The Internet, best known by most users as the World-Wide-Web, continues to expand at an amazing pace. We propose a new infrastructure to harness the combined resources, such as CPU cycles or disk storage, and make them available to everyone interested. This infrastructure has the potential for solving parallel supercomputing applications involving thousands of cooperating components. Our approach is based on recent advances in Internet connectivity and the implementation of safe distributed computing embodied in languages such as Java. We developed a prototype of a global computing infrastructure, called SuperWeb, that consists of hosts, brokers and clients. Hosts register a fraction of their computing resources (CPU time, memory, bandwidth, disk space) with resource brokers. Client computations are then mapped by the broker onto the registered resources. We examine an economic model for trading computing resources, and discuss several technical challenges associated with such a global computing environment.
@cite_1: Evolutionary algorithms are increasingly being applied to problems that are too computationally expensive to run on a single personal computer due to costly fitness function evaluations and or large numbers of fitness evaluations. Here, we introduce the Seamless Peer And Cloud Evolution (SPACE) framework, which leverages bleeding edge web technologies to allow the computational resources necessary for running large scale evolutionary experiments to be made available to amateur and professional researchers alike, in a scalable and cost-effective manner, directly from their web browsers. The SPACE framework accomplishes this by distributing fitness evaluations across a heterogeneous pool of cloud compute nodes and peer computers. As a proof of concept, this framework has been attached to the RoboGen open-source platform for the co-evolution of robot bodies and brains, but importantly the framework has been built in a modular fashion such that it can be easily coupled with other evolutionary computation systems.
@cite_32: In a connected world, spare CPU cycles are up for grabs, if you only make its obtention easy enough. In this paper we present a distributed evolutionary computation system that uses the computational capabilities of the ubiquituous Web browser. Asynchronous Javascript and JSON (Javascript object notation, a serialization protocol) allows anybody with a Web browser (that is, mostly everybody connected to the Internet) to participate in a genetic algorithm experiment with little effort, or none at all. Since, in this case, computing becomes a social activity and is inherently impredictable, in this paper we will explore the performance of this kind of virtual computer by solving simple problems such as the royal road function and analyzing how many machines and evaluations it yields. We will also examine possible performance bottlenecks and how to solve them, and, finally, issue some advice on how to set up this kind of experiments to maximize turnout and, thus, performance. The experiments show that we we can obtain high, and to a certain point, reliable performance from volunteer computing based on AJAJ, with speedups of up to several (averaged) machines.
@cite_3: The paper presents a distributed computing system that is based on evolutionary algorithms and utilizing a web browser on a client's side. Evolutionary algorithm is coded in JavaScript language embedded in a web page sent to the client. The code is optimized with regards to the memory usage and communication efficiency between the server and the clients. The server side is also based on JavaScript language, as node.js server was applied. The proposed system has been tested on the basis of permutation flowshop scheduling problem, one of the most popular optimization benchmarks for heuristics studied in the literature. The results have shown, that the system scales quite smoothly, taking additional advantage of local search algorithm executed by some clients.
@cite_34: In this article, the authors propose a new approach to distributed computing with Web browsers and introduce the WeevilScout prototype framework. The proliferation of Web browsers and the performance gains being achieved by current JavaScript virtual machines raises the question whether Internet browsers can become yet another middleware for distributed computing. With 2 billion users online, computing through Internet browsers has the potential to amass immense resources, thus transforming the Internet into a distributed computer ideal for common classes of distributed scientific applications such as parametric studies. As a proof of concept, the authors demonstrate how a cluster of globally distributed Internet browsers is used to compute thousands of bio-informatics tasks.
@cite_0: Abstract Project Bayanihan is developing the idea of volunteer computing , which seeks to enable people to form very large parallel computing networks very quickly by using ubiquitous and easy-to-use technologies such as web browsers and Java. By utilizing Java’s object-oriented features, we have built a flexible software framework that makes it easy for programmers to write different volunteer computing applications, while allowing researchers to study and develop the underlying mechanisms behind them. In this paper, we show how we have used this framework to write master-worker style applications, and to develop approaches to the problems of programming interface, adaptive parallelism, fault-tolerance, computational security, scalability, and user interface design.
@cite_19: With few exceptions, the field of Machine Learning (ML) research has largely ignored the browser as a computational engine. Beyond an educational resource for ML, the browser has vast potential to not only improve the state-of-the-art in ML research, but also, inexpensively and on a massive scale, to bring sophisticated ML learning and prediction to the public at large. This paper introduces MLitB, a prototype ML framework written entirely in Javascript, capable of performing large-scale distributed computing with heterogeneous classes of devices. The development of MLitB has been driven by several underlying objectives whose aim is to make ML learning and usage ubiquitous (by using ubiquitous compute devices), cheap and eff ortlessly distributed, and collaborative. This is achieved by allowing every internet capable device to run training algorithms and predictive models with no software installation and by saving models in universally readable formats. Our prototype library is capable of training deep neural networks with synchronized, distributed stochastic gradient descent. MLitB off ers several important opportunities for novel ML research, including: development of distributed learning algorithms, advancement of web GPU algorithms, novel field and mobile applications, privacy preserving computing, and green grid-computing. MLitB is available as open source software.
@cite_23: As an accessible and high-performance computing environment, we have studied Web-based Volunteer Computing (VC), in which users can join a VC with just accessing a specified URL on Web browsers. A major problem of Web-based VC has been the difficulty of implementing some advanced functions such as direct communications among users and file sharing, because all processes must be handled onWeb browsers. For this problem, we propose a solution to realize a P2P communication function among Web browsers by using recent Web techniques such as Web Real-Time Communication (WebRTC) API and its library, PeerJS. As an useful application of our method, we demonstrate file sharing on Web browsers. Experimental result indicates that our method achieves almost 50 reduction in the total time of file distribution.
@cite_15: In recent years, some scientists have embraced the distributed computing paradigm. As experiments and simulations demand ever more computing power, coordinating the eorts of many dierent processors is often the only reasonable resort. We developed an open-source distributed computing framework based on web technologies, and named it Capataz. Acting as an HTTP server, web browsers running on many dierent devices can connect to it to contribute in the execution of distributed algorithms written in Javascript. Capataz takes advantage of architectures with many cores using web workers. This paper presents an improvement in Capataz’ usability and why it was needed. In previous experiments the total time of distributed algorithms proved to be susceptible to changes in the execution time of the jobs. The system now adapts by bundling jobs together if they are too simple. The computational experiment to test the solution is a brute force estimation of pi. The benchmark results show that by bundling jobs, the overall perfomance is greatly increased.
@cite_16: Abstract Parallel computing on local area networks is generally based on mechanisms that specifically target the properties of the local area network environment. However, these mechanisms do not effectively extend to wide area networks due to issues such as heterogeneity, security, and administrative boundaries. We present a system which enables application programmers to write parallel programs in Java and allows Java-capable browsers to execute parallel tasks. It comprises a virtual machine model which isolates the program from the execution environment, and a runtime system realizing this virtual machine on the Web. Load balancing and fault masking are transparently provided by the runtime system.
@cite_13: The JAVM (Java Astra Virtual Machine) project is about harnessing the immense computational resource available in the Internet for parallel processing. In this paper, the suitability of Java for Internet-based parallel computing is explored. Next, existing implementations of systems that make use of Java for network parallel computing are presented and categorized. A critique of these implementations follows. Basing on the critique, the requirements and goals of an effective parallel computing system in the Internet environment are singled out. These serve as the blueprint for the development of the JAVM system. Its infrastructure and features, namely ease of use, heterogeneity, portability, security, fault tolerance, load balancing, scalability and accountability, are discussed. Lastly, experimental results based on the running of several parallel applications in the JAVM environment are presented. Basing on the results, the kind of parallel applications that would be well suited for running in JAVM are identified.
@cite_20: The paper presents a new, cost eﬀective, volunteer computing based platform.It utilizes volunteers’ web browsers as computational nodes. The computationaltasks are delegated to the browsers and executed in the background (indepen-dently of any user interface scripts) making use of the HTML5 web workerstechnology. The capabilities of the platform have been proved by experimentsperformed in a wide range of numbers of computational nodes (1–400).
@cite_11: JSMapReduce is an implementation of MapReduce which exploits the computing power available in the computers of the users of a web platform by giving tasks to the JavaScript engines of their web browsers. This article describes the implementation of JSMapReduce exploiting HTML 5 features, the heuristics it uses for distributing tasks to workers, and reports on an experimental evaluation of JSMapReduce.
Compared to other published volunteer computing tools, we are the first to have successfully tested with a thousand participants and the first to use WebRTC to connect participants in a fat-tree overlay. Most published volunteer computing tools @cite_29 @cite_16 @cite_13 @cite_0 @cite_3 @cite_15 @cite_34 @cite_23 @cite_1 were tested with less than a hundred of participants. Some of the most recent have been tested with more than a hundred participants @cite_32 @cite_11 @cite_19 and even up to 400 concurrent participants @cite_20 . But the largest internet deployments of custom tools @cite_32 @cite_11 @cite_19 have so far reached a hundred concurrent participants @cite_19 .