automatic differentiation in deep learning symbolic differentiation symbolic differentiation in its basic form is set of symbol rewriting rules applied to the loss function to arrive at the derivatives/gradients consider two of such simple rules dx dx dx and nx dx given function such as ( we can successively apply the the symbol writing rules to first arrive at dx dx by applying the first rewriting ruleand by applying the second rule symbolic differentiation is thus automating what we do when we derive gradients manually of coursethe number of such rules can be largeand more sophisticated algorithms can be leveraged to make this symbol rewriting more efficient in its essencehoweversymbolic differentiation is simply the application of set of symbol rewriting rules the key advantage of symbolic differentiation is that it generates legible mathematical expression for the derivative/gradient that can be understood and analyzed the key problem with symbolic differentiation is that it is limited to the symbolic differentiation rules already definedwhich can cause us to hit roadblocks when trying to minimize complicated loss functions