How do optical illusions work (photo mosaic) from a signal processing perspective?
What are the signal processing principles behind our eye merging the many tiny images into a large coherent one?


From what I've read (Source 1: Front-End Vision and Multi-Scale Image Analysis, Source 2: Information Visualization), our eyes apply something very similar to a set of gabor filters to the visual inputs we get. So if you, i.e. focus on Mona Lisa's eye, your brain processes gabor filters at the scale roughly the size of the eye, and such a filter removes details at smaller scales.


  Are there etiological explanations of optical illusions in general - by this I mean can it be explained why image inputs get parsed in 'wrong' ways by the eye/brain, why does the human parser get confused; what transfer function is in play?


Regarding optical illusions in general: There is tons of research. The two books I've mentioned above quote some of it, but I don't know enough about it to compress it into a good answer here.

But regarding this special case, there's actually a very elegant explanation in Front-End Vision and Multi-Scale Image Analysis, p208, which I'll try to repeat here: Take a "natural" sample image. Split the image into small blocks. Multiply each block with a window function (e.g. a gaussian).

For example, I'll take the "Lena" image in Mathematica:

sample = ColorConvert[ExampleData[{"TestImage", "Lena"}], "Grayscale"];


split it into 16x16 pixel blocks:

blocks = Flatten[Partition[ImageData[sample], {16, 16}, {5, 5}], 1];


multiply each block with a gaussian window:

window = Table[Exp[-(x^2 + y^2)/25], {x, -7.5, 7.5}, {y, -7.5, 7.5}];
blockTimesWindow = Map[window * # &amp;, blocks];


the blocks look something like this:



Next I'll use SVD to find the principal components of these blocks:

{u, s, v} = SingularValueDecomposition[Flatten /@ blockTimesWindow, 10];
Image[0.5 + Partition[#, 16], ImageSize -&gt; 100] &amp; /@ Transpose[v]


and visualize them:



These are the principal components of small parts of my test image. Or in other words: any small part of my test image is (approximately) a weighted sum of these principal components, plus noise. And it's not just this test image. Take any photograph of a natural scene, the result will look very similar.

As you can see, the principal components look suspiciously like gabor filter kernels or gaussian derivatives. That explains why filters like these are a good choice to analyze natural images: in a natural image, any small part is (almost) the weighted sum of a few gabor filter kernels plus noise. And it explains why evolution would choose this kind of low-level processing.

Now your image is not a natural image: Neighboring pixels are not correlated the way they would be in a natural image. But our eyes still use processes that were optimized for natural scenes, where differences at smaller scales don't carry as much important information, so they are filtered out. That explains why the illusion works.