When using pretrained model(vgg, resnet like) as backbone, should we use it in `eval mode` or in `train mode`?
However, in the general case, if you are freezing the model (with requires_grad = False) then you are not updating the running statistics anymore and should therefore use the running statistics, i.e. put the model in eval mode.
VGG's backbone does not have any normalization layers nor dropouts. So in the end it does not matter whether you put the backbone into eval or training mode.
When we use a pretrained model, e.g. vgg16, as a backbone of the whole model which plays as a feature extractor, the model's data flow can be depicted as below:
Data --> vgg16 --> another network --> output
As for now, I've set False require_grads flags for all parameters in vgg16, and exclude those parameters from my optimizer's param list, so the vgg16 will not be modified during the training period.
But when I step further in my study, I'm now wondering which mode should vgg16 be used in? Should we call vgg16.eval() before running training epochs?