Nutritional image classification task I need a model that is able to receive as input an image of a nutritional information chart and tell the level of sugar that the product has. It would be a 3-class classification problem (low if sugar is below 5g, medium if it's between 5 and 22.5g and high if it has more than 22.5g). I have prepared all the data and I have 16000 images in total. However, I'm not able to train a proper model with the data. I have tried a simple convolutional neural network of 3 convolutional layers, the pretrained inception resnet v2 from keras, and even an attentional model (Github). The result is always the same, an accuracy equal to the proportion of samples from the most common class. So these models are unable to solve the issue and just bet for the most likely.
What kind of network could be able to solve this problem? I have never dealt with networks that have to &quot;read&quot; and classify text.
Another option would be to apply an established Optical Character Recognition (OCR) system to the raw images. After converting the raw images to plain text, put the relevant data into a tabular dataframe. Once in a tabular dataframe, fit a deep learning or traditional machine learning model can be fit.
It can be tempting to build an end-to-end deep learning system. However, some problems are best solved by combination of subsystems. Those subsystems could include human expertise, rules, traditional machine learning, and just a small amount of deep learning.