Classification Model based on Ordered Features I am trying to build a classifier for a specific card dataset let's say cards or no cards. I am using Mobilenet trained on Imagenet dataset as my classifier and further training it on my dataset. I am able to train it, and it's performance is quite good on the dataset. Let's say my card has for different regions of interest as shown below:-


It is able to perfectly recognize the above passed image as card.

But I am able to fool my classifier if I change the ordering of the regions of card and then giving it to classifier as shown below:-


I want the classifier to learn the exact occurrences of features of cards as in First Image so that if I alter them it should be able to reject that card (like in second image).

Is there any way to achieve this?
You could try a recurrent neural network classifier.

Alternatively, and perhaps more easily, augment your dataset to contain images of cards with changed ordering of the card images, and train on the resulting dataset.