Why the resnet110 I train on CIFAR10 dataset only get 77% test acc
I trained the Resnet110 on CIFAR10 dataset, and I got 100% acc on training, but only 77.85% on test dataset. What is the problem probally be?
Otherwise, I use Pytorch framwork.
Thank U very much!

    ------------------------------------------------------------------------
Train Epoch: 200 [0/50000 (0%)] Loss: 0.000811, Accuracy: 100.00
Train Epoch: 200 [12800/50000 (26%)]    Loss: 0.000335, Accuracy: 100.00
Train Epoch: 200 [25600/50000 (51%)]    Loss: 0.000757, Accuracy: 100.00
Train Epoch: 200 [38400/50000 (77%)]    Loss: 0.000334, Accuracy: 100.00
Epoch time:45.98s
Accuracy of plane : 81 %
Accuracy of   car : 88 %
Accuracy of  bird : 65 %
Accuracy of   cat : 60 %
Accuracy of  deer : 73 %
Accuracy of   dog : 69 %    
Accuracy of  frog : 81 %
Accuracy of horse : 82 %
Accuracy of  ship : 86 %
Accuracy of truck : 86 %

Test set: Average loss: 1.3605, Accuracy: 77.58%


ResNet-101 is definitely too big for CIFAR10, go with smaller versions, ResNet-18 from torchvision should be fine.

Furthermore, you could train those really fast using super convergence (e.g. setting learning rate to 5 or 3), see this article or other related. You could do so in 18 epochs or so with torch.optim.AdamW I think. Furthermore, such high learning rate plays a regularizing role as it will only converge to really flat minima regions.

In order to not overfit, use really powerful image augmentation, rotations, flipping, CutOut, maybe MixUp. You could find them inside albumentations third party library.
