What's the difference between transfer learning and feature extraction in CNN?
Transfer learning is, as you said, retraining the final layer of a deep network. Not only is this useful for solving problems with limited training examples, but also when you don't have adequate computing resources to train a network from scratch. Some models have hundreds of millions of parameters, which could take weeks to train if you only have modest equipment. However, if you have sufficient data, adapting weights via transfer learning is not preferable because the features that were extracted from the original training process are unlikely to be ideal for another application.
Feature extraction, in the context of CNNs, is not necessarily an explicit process, rather a sort of high-level product of the training process. Feature extraction refers to the portion of the training process by which a CNN learns to map input space to a latent space that can subsequently be used for classification via the final layer. In other words, the hidden layers learn discriminatory features in the form of weight-adjusted (usually by backpropagating the error) convolutional filters. Thus the term &quot;feature extraction&quot; generally refers to the portion of the training process that occurs before the final layer. So it is not part of transfer learning in which only the last layer is trained.