Visualizing Distributed Systems without a Partial Order

Distributed system behavior can be visualized without representing the partial ordering of events. Theia [43] displays a visual signature that summarizes various aspects of a Hadoop execution, such as the execution’s resource utilization. Similarly, shared resource use can be modeled and visualized to represent complex system behavior, such as MapReduce and WatsonDeepQA [31]. SALSA [116] and Mochi [117] are log analysis tools that extract and visualize the control- and data-flow of Hadoop applications’ logic in a variety of forms, including finite state machines. These tools can be used for failure diagnosis and performance debugging. Overall, tools in this category necessarily provide high-level summaries of a system’s behavior. ViVA [73] has a similar goal as ShiViz, but its focus is on visualizing component interactions in a distributed setting; ViVA neither

A promising alternative approach is to use NLP techniques to reconstruct distributed workflow information from existing logs, e.g., as in recent work by Pi et al. [90]. This approach does not

ACM Transactions on Software Engineering and Methodology, Vol. 29, No. 2, Article 9. Pub. date: March 2020.

Tracing and Debugging Distributed Systems

A variety of tracing systems have been proposed to track requests in distributed systems, such as Dapper [110], Zipkin [125], X-Trace [39], and others [3, 5, 100, 115]. For example, vPath [115] is a VM monitor for understanding causality by tracking thread and network activity, and WAP5 [103] relinks a target application to custom system libraries to monitor network communication and reconstruct request paths using statistical inference. By contrast, XVector uses vector clocks to

ShiViz’s side-by-side view with differencing is similar to the side-by-side view in work to evalu- ate the visualization strategies for distributed request-flow comparison by Sambasivan et al. [103]. However, request flow visualization is fundamentally different from observing the partial order- ing of all events in the system. In particular, a single request flow rarely has the same amount of concurrency. More recent work on differencing execution traces [114] collapses loops and clusters traces; adding these features to ShiViz is likely useful.

XVector adds useful partial ordering information to execution logs. We have previously used this information to mine temporal properties to relate events at different hosts and to infer a model of the distributed system from multiple executions. These models, too, were useful for understanding system behavior [10, 11].

Reproducing a bug is a key aspect of debugging. Unfortunately, reproducing bugs in distributed systems can be challenging due to sources of nondeterminism, including scheduling and paral- lel execution. Friday [44] helps replay distributed system executions, eliminating some sources of nondeterminism. Recon [72] can further aid debugging by collecting relations between dis- tributed system artifacts, such as hosts, communication channels, events, and so on, and enabling querying these relationships after the execution. The goal of ShiViz is to improve understand- ing of a distributed system execution, which is often a necessary step in debugging distributed systems. ShiViz does not directly facilitate execution replay, although the happens-before rela- tionship XVector logs can help reconstruct what happened in a non-deterministic execution. The work on debugging distributed systems is complementary to ShiViz, and these tools may benefit

DistIA [17] is a dynamic impact analysis approach for distributed systems. This tool does not visualize/explain executions as ShiViz does; instead, DistIA helps developers make sense of how changes to their code would influence the execution of the processes in the distributed system.

Summarizing Logs and System Behavior

Numerous techniques attempt to summarize multiple system executions as a behavioral model. These approaches target both sequential executions [9, 12, 15, 22, 35, 41, 45, 62, 63, 71, 74, 77, 84, 98, 106, 119] as well as concurrent executions [2, 10, 64]. Such summaries can be created remotely and used to facilitate code reuse [78, 79]. The goal of summarization is to abstract the observed behavior, which is necessarily partial, into a compact representation that often includes other, unobserved, but likely behavior. As a result, while these techniques strive to abstract the behavior precisely, they often make overzealous abstraction decisions that result in loss of precision [62, 71]. Unlike these summarization techniques, ShiViz focuses on visualizing executions exactly as they occurred. ShiViz does not attempt to predict unobserved behavior.

ACM Transactions on Software Engineering and Methodology, Vol. 29, No. 2, Article 9. Pub. date: March 2020.

Performance-based Analysis

Distributed system visualization can help understand system performance, and many existing tools aim to analyze or visualize performance-related aspects of distributed system behavior [58, 59]. Coz [23] finds causal performance relationships between execution blocks in concurrent systems. Coz can be used to identify how speeding up a particular block or set of blocks will affect system running time. Unlike ShiViz, Coz does not visualize system executions but does help developers

Flame graphs [48, 49] visualize profiler output to help developers reason about how is consuming resources, which can speed up root cause analysis. Flame graphs combine elements using a resource, using element hierarchy to visualize those elements that use the resources. These graphs can be explored interactively to understand resource use.

Integrated Development Environments