Image multi class classifier CNN I have a problem, im designing a multiclass classifier to classify medic images, I have to classify in which grade of desease is it, this are 6 grades , each time the joint deforms a little, so, mi original dataset was imabalanced, each class have like 16 to 200  images that are very very similar with some little deatils of change, ,so i did some contrasts and flips to the images to have like 500 images per class aprox, i used a vgg16 architechture to train the model but, it soesnt work 30% acc, i ddid a simple model with 2 layers of convolution, I have like 70% of accurracy but my recall is very very low like 0.001 to 0.3, and when I predict with an image, all the images goes to the incorrect class, i dont know how to correct my model , use other metrics or some architechture for this type of images?.Here's a sample of the code https://github.com/Franciscogtu/OARSI-IMAGE-CLASSIFIER.git
Thank you for the help
Have you tried a batch normalization layer between the conv2D layers and the dense top? I'd also keep the number of filters low ,maybe 32 or 64. Same for the number of nodes in the top, no point in having a huge dense top with hundreds of nodes.