How to reduce overfitting in a pre-trained network I have a custom dataset with 10 classes and I am using a pre-trained resnet18 model from torch-vision. I can clearly see it's over-fitting because: the model is trained for 75 epochs with a batch size of 4 and from epoch 30 the validation accuracy stops increasing and the training accuracy keeps increasing.
Things I did to improve generalisation performance and reduce overfitting:

I normalized my data with a calculated mean and std on my training data.
I added random rotation, random horizontal flip and random vertical flip as data augmentations.

The above were the results after all of these, I can't think of other regularization techniques that produce regularization effects without changing the architecture of the model which will completely contradict the purpose of a pre-trained network, and I am using one because I only have around 300 images in my each class.
In transfer learning there are two parameters which influence the basic setup to go for:

Size of new dataset
Similarity of new dataset to dataset of pre-trained model

When your dataset is small the problem is that high capacity pre-trained models can easily overfit if you re-train too many layers. And since you re-trained multiple layers this could be an issue here.
Instead, try the following two options:

Re-train only the last fully connected layer. This is the general approach if your data is similar to the data which the pre-trained model was trained on.
Remove the conv. layers towards the end of the pre-trained model and re-train only the new fully connected layer. For ResNet18 you could try tossing conv4 and 5, for example. This is the general approach in transfer learning for small datasets if your data is not very similar to the original data and you only want to use layers for lower level features since the datasets do not have similar higher level features.

If you are interested in a theoretical reading on the topic, the paper &quot;How transferable are features in deep neural networks?&quot; could be of interest.