before we beginthe mathematical building blocks of neural networks called backpropagation (also sometimes called reverse-mode differentiationbackpropagation starts with the final loss value and works backward from the top layers to the bottom layersapplying the chain rule to compute the contribution that each parameter had in the loss value nowadaysand for years to comepeople will implement networks in modern frameworks that are capable of symbolic differentiationsuch as tensorflow this means thatgiven chain of operations with known derivativethey can compute gradient function for the chain (by applying the chain rulethat maps network parameter values to gradient values when you have access to such functionthe backward pass is reduced to call to this gradient function thanks to symbolic differentiationyou'll never have to implement the backpropagation algorithm by hand for this reasonwe won' waste your time and your focus on deriving the exact formulation of the backpropagation algorithm in these pages all you need is good understanding of how gradient-based optimization works licensed to