The Internet of Things (IoT) is offering unprecedented observational data that are used for managing Smart City utilities. Edge and Fog gateway devices are an integral part of IoT deployments to acquire real-time data and enact controls. Recently, Edge-computing is emerging as first-class paradigm to complement Cloud-centric analytics. But a key limitation is the lack of a platform-as-a-service for applications spanning Edge and Cloud. Here, we propose ( ECHO ), an orchestration platform for dataflows across distributed resources. ( ECHO ) ’s hybrid dataflow composition can operate on diverse data models – streams, micro-batches and files, and interface with native runtime engines like TensorFlow and Storm to execute them. It manages the application’s lifecycle, including container-based deployment and a registry for state management. ( ECHO ) can schedule the dataflow on different Edge, Fog and Cloud resources, and also perform dynamic task migration between resources. We validate the ( ECHO ) platform for executing video analytics and sensor streams for Smart Traffic and Smart Utility applications on Raspberry Pi, NVidia TX1, ARM64 and Azure Cloud VM resources, and present our results.
@cite_15: Task mapping, which basically consists of mapping a set of tasks onto a set of nodes, is a well-known problem in distributed computing research. As a particular case of distributed systems, the Internet of Things (IoT) poses a set of renewed challenges, because of its scale, heterogeneity and properties traditionally associated with wireless sensor networks (WSN), shared sensing, continous processing and real time computing. To handle IoT features, we present a formalization of the task mapping problem that captures the varying consumption of resources and various constraints (location, capabilities, QoS) in order to compute a mapping that guarantees the lifetime of the concurrent tasks inside the network and the fair allocation of tasks among the nodes. It results in a binary programming problem for which we provide an efficient heuristic that allows its resolution in polynomial time. Our experiments show that our heuristic: (i) gives solutions that are close to optimal and (ii) can be implemented on reasonably powerful Things and performed directly within the network, without requiring any centralized infrastructure.
@cite_14: Internet of Things (IoT) domains generate large volumes of high-velocity event streams from sensors, which need to be analyzed with low latency to drive decisions. Complex Event Processing (CEP) is a Big Data technique to enable such analytics and is traditionally performed on Cloud Virtual Machines (VM). Leveraging captive IoT edge resources in combination with Cloud VMs can offer better performance, flexibility, and monetary costs for CEP. Here, we formulate an optimization problem for energy-aware placement of CEP queries, composed as an analytics dataflow, across a collection of edge and Cloud resources, with the goal of minimizing the end-to-end latency for the dataflow. We propose a Genetic Algorithm (GA) meta-heuristic to solve this problem and compare it against a brute-force optimal algorithm (BF). We perform detailed real-world benchmarks on the compute, network, and energy capacity of edge and Cloud resources. These results are used to define a realistic and comprehensive simulation study that validates the BF and GA solutions for 45 diverse CEP dataflows, LAN and WAN setup, and different edge resource availability. We compare the GA and BF solutions against random and Cloud-only baselines for different configurations for a total of 1,764 simulation runs. Our study shows that GA is within 97 of the optimal BF solution that takes hours, maps dataflows with 4--50 queries in 1--26s, and only fails to offer a feasible solution ≤20 of the time.
@cite_13: By bridging the physical and the virtual worlds, the Internet of Things (IoT) impacts a multitude of application domains, among which smart cities, smart factories, resource management, intelligent transportation, health and well-being to name a few. However, leveraging the IoT within software applications raises tremendous challenges from the networking up to the application layers, in particular due to the ultra-large scale, the extreme heterogeneity and the dynamics of the IoT. This paper more specifically explores how the service-oriented architecture paradigm may be revisited to address challenges posed by the IoT for the development of distributed applications. Drawing from our past and ongoing work within the MiMove team at Inria Paris, the paper discusses the evolution of the supporting middleware solutions spanning the introduction of: probabilistic protocols to face scale, cross-paradigm interactions to face heterogeneity, and streaming-based interactions to support the inherent sensing functionality brought in by the IoT.
IoT Middleware is an active research area as well. The @cite_13 has proposed an SOA architecture for mobile IoT, with a focus on the functional scalability. A novel probablistic registry allows low-latency approximate queries for registered sensing and actuation services. It does static scheduling of streaming service dataflows using the Dioptase middleware @cite_15 , and interfacing across heterogeneous IoT protocols using an Enterprise Service Bus (ESB). in contrast supports hybrid data models -- a higher level abstraction than protocols, offers richer composition semantics including delegating to external engines, and uses point-to-point communication between tasks (push and pull) rather than a central ESB. Our Hypercat registry is simpler but based on BSI standards, and can be replaced or federated for scaling. Advanced scheduling algorithms @cite_14 or device mobility is not a focus in this paper, but future work.