Model not learning when using transfer learning
Well the critical step for finetuning/transfer learning any model in tensorflow[1,2+]:

use right preprocessing for images (find the preprocessing which requires mobilenetv2)
start with &quot;imagenet&quot; weights
change the BatchNormalization behavior, in tf the default momentum value is 0.99 which is not best, in pytorch it is set to 0.9 which is better when finetunig your model and you don't have lot of data (see the link and this for more)