Title: Tangent: Automatic differentiation using source-code transformation for dynamically typed array programming,
Abstract: The need to efficiently calculate first- and higher-order derivatives of
increasingly complex models expressed in Python has stressed or exceeded the
capabilities of available tools. In this work, we explore techniques from the
field of automatic differentiation (AD) that can give researchers expressive
power, performance and strong usability. These include source-code
transformation (SCT), flexible gradient surgery, efficient in-place array
operations, higher-order derivatives as well as mixing of forward and reverse
mode AD. We implement and demonstrate these ideas in the Tangent software
library for Python, the first AD framework for a dynamic language that uses
SCT.