Can we apply transfer learning between any two different CNN architectures?
No, transfer learning cannot be applied &quot;between&quot; different architectures, as transfer learning is the practice of taking a neural network that has already been trained on one task and retraining it on another task with the same input modality, which means that only the weights (and other trainable parameters) of the network change during transfer learning but not the architecture.
In my understanding, transfer learning is also only really effective in deep learning, but I could be wrong, considering that this Google search seems to yield some results.
You might otherwise be thinking of knowledge distillation, which is a related but different concept, where an already trained network acts as a teacher and teaches another network (a student network) with possibly a different architecture (or a machine learning model not based on neural networks at all) the correct outputs for a bunch of input examples.