What model structure I should use to train on low res and blurry images?
Yes, you may use the small model you have but you need to tune it.
The models you have used are not too big if any overfitting happens you can just use dropout to alleviate that.
You need to improve the quality of your images. One way is to reconstruct them using autoencoders or GANs (Super-resolution GANs). Then, you can classify them.