How to use MNIST dataset to make predictions on similar images (colorblindness charts)?
I think the $RGB$ color components of the pixels form vectors that if you see them in spherical coordinates, angles are colors, and radius is intensity.

I tried it and ended up with:



Build a 3-class classifier for the angles calculated for each pixel (background and two colors).

If you end up with only 2 classes (the third one has very few members), you have an intensity problem, and you should reclassify for radius (or just take a threshold value).

If you have 3 classes, you have a color problem. Drop the class of the (black) background color (probably just the class of the top left pixel. Also drop the class with the largest mean distance to the center of the image, the second background.

The remaining pixels will be white, the others black.

You still have a picture with circles forming the digit, but a pooling filter will even it out.



3-way classification directly on the image was unsuccessful, because the distance between light gray and dark gray is similar to light gray/light purple. The angles are much more separable.