Why the performance of VGG-16 is better than Inception V3? I’ve initialised VGG16 and InceptionV3 with ImageNet weights and fine tuned using very small learning rate from the first layer on a chest X-ray dataset of 20K images (since the medical images are different from ImageNet classes). I reportedly find the VGG16 shows no overfitting and is more accurate than InceptionV3 that overfits and gives less accurate results. What is the reason?
What are the cons?
It has so many weight parameters, the models are very heavy, 550 MB + of weight size.
Which also means long inference time
Why not just make the model deeper?
More heavier model
More training time
Vanishing gradient problem

With a given receptive field(the effective area size of input image on which output depends), multiple stacked smaller size kernel is better than the one with a larger size kernel because multiple non-linear layers increases the depth of the network which enables it to learn more complex features, and that too at a lower cost. 3X3 kernels help in retaining finer level properties of the image.