Using neural network for "features matching" binary classification
From what I understand, your dataset is of pairs of images and a binary classification of their pairing?

There are models using an architecture named Siamese Neural Networks which are used exactly for this task of determining similarity.

You can start by reading the following article: Learning to Compare Image Patches via Convolutional Neural Networks. They are using Siamese architecture to compare between different image patches (similar to what you are describing):