How does backpropagation differ from reverse-mode autodiff Going through this book, I am familiar with the following:


  For each training instance the backpropagation algorithm first makes a
  prediction (forward pass), measures the error, then goes through each
  layer in reverse to measure the error contribution from each
  connection (reverse pass), and finally slightly tweaks the connection
  weights to reduce the error.


However I am not sure how this differs from the reverse-mode autodiff implementation by TensorFlow. As far as I know the above algorithm first goes through the graph in the forward direction and then in the second pass computes all partial derivatives for the outputs with respect to the inputs. This is very similar to the propagation algorithm.

How does backpropagation differ from reverse-mode autodiff ?
They are more or less specify the same thing. I think the difference is that back-propagation refers to the updating of weights with respect to their gradient to minimize a function; "back-propagating the gradients" is a typical term used. 

Conversely, reverse-mode diff merely means calculating the gradient of a function. Nothing here in this process implies that we aim to the weights/values of a function.