Detecting original vs. edited (reposted / recompressed) image I'm trying to create something to help solve this problem:

My goal is that, given two images where one is an edit of the other, produce a system that outputs which one is most likely to be the original.
I have attempted to solve the problem in the following way.

Collect a data set of random images that one may find in their social media blog. (10,000 images)

Apply random lossy transformations to the image: resizing (with various scaling algorithms), cropping, lossy compression (JPEG)

Preprocess input into training data:

Metadata:

Collect file size, resolution, file format, lossy compression quality

Add jitter (to prevent overfitting by recognizing specific images), jitter varies per 8x8 sample

Output is a fixed number of floats per datum



Input pixels:

Break the image into 8x8 chunks (the size is chosen to coincide with JPEG MCU size)

Select a fixed number of chunks with the most entropy (changes in pixels)

Select a color channel

Divide by 255.0

Output is 64 floats (one per pixel)



DCT values:

Using the pixel data from above, perform a DCT transform

Take the absolute value of the DCT value

Output is 64 floats





Feed the above data into a neural network.
I don't know if it makes sense or not, but I started with a straight-forward model and did a simple hill-climbing hypersearch which added/removed/edited layers, and it produced the following model:

Each sample (and associated metadata) is fed as a single input.

Fit the model to the labels 0.0 or 1.0, with 0.0 representing original (less noise) and 1.0 representing edited (more noise).
Even though these labels don't describe any quality of the image itself, the goal is to use them to indicate some kind of ordering. The idea here is to &quot;pull&quot; the weights towards 0 for images known to have less noise and towards 1 for those with more noise, which I hope will cause the model to output higher values for images which have been edited.


Currently the model's accuracy is 88% (i.e. it correctly produces higher values for our edited versions for 88% of input image pairs).
My question is if there is a better way to represent ordering in this case other than labeling pairs 0 and 1. One potential issue with this is that the model is penalized if it outputs a score &gt;1 or &lt;0 (though currently avoided using sigmoid activation if that makes sense). Also, if there are other obvious improvements in the whole process. Thank you!
My question is if there is a better way to represent ordering in this case other than labeling pairs 0 and 1.

I solved this by creating an artificial comparator layer, which takes two inputs, and outputs 0 if the first is smaller or 1 if the first is greater. This allows backpropagation to correctly push the two away from each other within the same batch.

Also, if there are other obvious improvements in the whole process.

See my other question: Producing a confidence output to use in a weighted average layer