How does the bounding box regressor work in Fast R-CNN?
A very clear and in-depth explanation is provided by the slow R-CNN paper by Author(Girshick et. al) on page 12: C. Bounding-box regression and I simply paste here for quick reading:






  Moreover, the author took inspiration from an earlier paper and talked about the difference in the two techniques is below:





  After which in Fast-RCNN paper which you referenced to, the author changed the loss function for BB regression task from regularized least squares(ridge regression) to smooth L1 which is less sensitive to outliers!. Also, you embed this smooth L1 loss in the multi-task loss function so that we can jointly train for classification and bounding-box regression that wasn't done before in R-CNN or SPP-net!





  However, the same author has changed the loss function again in the upcoming paper faster-RCNN
  Later, in FCN
  Many a time, in order to learn about a topic, you need to do backtracking through research papers! :) Hope it helps!