Improvements to video-game cover CNN classifier (keras)
Here are some points to consider:


  1) I've written a Metacritic Scrubber which takes PS2, PS3, PS4, Xbox360 and Xbox One games which have metacritic scores, downloads the artwork and labels it with the score. Duplicate games are removed (which may be a mistake since they sometimes have different scores based on platform). I'm pretty happy with this code.


Good!


  2) I've rounded scores on a scale of 0-9 instead of 0-100 to make the number of classifications smaller.


This also should be fine.


  3) images are 123x98 with 3 channels. All images have been stretched to this size. I wonder if this may be a source of problems because some covers have been stretched. Values are between 0-255 for each channel.


I would normalise the values between 0-1 (norm_values = values/255). Regarding the size of the files I don't think this should be a problem. For example, Cifar-10 are 32x32 or the average image resolution on ImageNet is 469x387 pixels although most approaches resize them to 256x256. I think your size should be fine.


  4) Games with square cover art (DLC, non-retail, etc) have been omitted.


Not much to add here.


  This leaves me with a data set of 3816 game covers. I figured this would probably be enough for an initial investigation.
  The model I've built has been based on the work by Iwana et al in this paper: https://arxiv.org/abs/1610.09204


Here is where I actually think we have some problems. First, your dataset is quite small. 3816 reduced to 1908 when splitting train/test... This is not good especially with such structure. In the paper you mentioned, they are using a network with around 2.5M parameters and they used ~137K samples. Your 1908 dataset seems tiny compared and you are using the same model structure... 

In my opinion your model is not able to do better, simply put. It doesn't matter what parameters you choose you need (a lot) more data. You might try reducing the size of your network, also create more data using noise addition, mirror samples, etc... and see if these help somehow.

Finally, apart from the size, you have no way to tell if your data is representative enough so that any model can learn from it. Therefore, 35% accuracy is as good as any other value I am afraid. Anyway, your mission seemed to be able to complete your experiment and you did. And you learnt a lot about limitations of deep models, so I'd say: good work!