automatic differentiation in deep learning numerical differentiation numerical differentiationin its basic formfollows from the definition of derivative/gradient it used to estimate the derivative of mathematical function derivate of with respect to more specifically defines the rate of change of with respect to simple way would be to compute the slope of the function through the line xf(xand +hf( +hsogiven that df  dx we can compute the '(xusing the forward difference method as dh setting suitably small value for similarlywe can compute '(xusing the backward difference method as againby setting suitably small value for more symmetric form is the central difference approachwhich computes as extrapolation is process of using known values to project value outside of the intended existing known range richardson extrapolation