Title: Deep Learning on Operational Facility Data Related to Large-Scale Distributed Area Scientific Workflows,
Abstract: Distributed computing platforms provide a robust mechanism to perform
large-scale computations by splitting the task and data among multiple
locations, possibly located thousands of miles apart geographically. Although
such distribution of resources can lead to benefits, it also comes with its
associated problems such as rampant duplication of file transfers increasing
congestion, long job completion times, unexpected site crashing, suboptimal
data transfer rates, unpredictable reliability in a time range, and suboptimal
usage of storage elements. In addition, each sub-system becomes a potential
failure node that can trigger system wide disruptions. In this vision paper, we
outline our approach to leveraging Deep Learning algorithms to discover
solutions to unique problems that arise in a system with computational
infrastructure that is spread over a wide area. The presented vision, motivated
by a real scientific use case from Belle II experiments, is to develop
multilayer neural networks to tackle forecasting, anomaly detection and
optimization challenges in a complex and distributed data movement environment.
Through this vision based on Deep Learning principles, we aim to achieve
reduced congestion events, faster file transfer rates, and enhanced site
reliability.