Fine Tuning the Neural Nets I have recently read about Fine Tuning, and what I want to know is, when we are fine-tuning our model is it necessary to Freeze the model and train only the top part of the model and then unfreeze some layers and again train the model or one can directly begin by unfreezing some layers? Till now, I have read that one does not unfreeze the layers directly because then we risk losing the important features captured by the earlier layers
It is a good practice to freeze the early layers when you finetune the model. There are two reasons why you want to do this :


Your new layers is initialized randomly and will always start with very big loss. Big loss means big gradient and if the weights are not frozen this will be propagated . Your model will become less stable and might fail to converge. To put it simply you don't want to put the blame of false prediction on the earlier layers.
You are saving a lot of computation resources. Training a neural network, is not a small issue, and training only a small section of the model will give you decent enough accuracy without expending too much computational resource.


Unfreezing the earlier layer is up to you on the later stage of training (Some people do some people don't) but if you feel you might need to do this to push for better performance, then just try it.