How to Apply the SSIM Measure on RGB Images? I want to compare the similarity of two images by looking at its structure and colors. I need a measure that takes both structural and color fidelity into account. When I checked the formulation of original paper, it is said only x and y mean and variance is measured. Does SSIM account for structure only or does it take color similarity into account too?
What usually can be done is calculate it per channel and have the average (Or weighted average).
So if you have $ {R}_{1}, {G}_{1}, {B}_{1} $ and $ {R}_{2}, {G}_{2}, {B}_{2} $ you would do:
$$ {SSIM}_{r} \left( {R}_{1}, {R}_{2} \right), {SSIM}_{g} \left( {G}_{1}, {G}_{2} \right), {SSIM}_{b} \left( {B}_{1}, {B}_{2} \right) $$
Then $ {SSIM}_{rgb} = {w}_{r} {SSIM}_{r} + {w}_{g} {SSIM}_{g} + {w}_{b} {SSIM}_{b} $.
For instance, MATLAB's ssim() does the above with $ {w}_{r} = {w}_{g} = {w}_{b} = \frac{1}{3} $.
SciKit Image's compare_ssim() also apply it channel wise using the multichannel flag: