Title: High-order asynchrony-tolerant finite difference schemes for partial differential equations,
Abstract: Synchronizations of processing elements (PEs) in massively parallel
simulations, which arise due to communication or load imbalances between PEs,
significantly affect the scalability of scientific applications. We have
recently proposed a method based on finite-difference schemes to solve partial
differential equations in an asynchronous fashion -- synchronization between
PEs is relaxed at a mathematical level. While standard schemes can maintain
their stability in the presence of asynchrony, their accuracy is drastically
affected. In this work, we present a general methodology to derive
asynchrony-tolerant (AT) finite difference schemes of arbitrary order of
accuracy, which can maintain their accuracy when synchronizations are relaxed.
We show that there are several choices available in selecting a stencil to
derive these schemes and discuss their effect on numerical and computational
performance. We provide a simple classification of schemes based on the stencil
and derive schemes that are representative of different classes. Their
numerical error is rigorously analyzed within a statistical framework to obtain
the overall accuracy of the solution. Results from numerical experiments are
used to validate the performance of the schemes.