Image processing: using PCA for 3d object recognition I read this lecture on using PCA/eigenfaces: http://www.cse.psu.edu/~rcollins/CSE486/lecture32.pdf

And I want to use this general procedure to recognize a 3D clay model that I took pictures of from multiple angles.

I'm having trouble conceptualizing how it's actually implemented in software. I've "trained" the initial mean and eigenvectors for using ~40 images offline. So I have new input images that will have my 3D model in it (or wont).

My confusion: I'm worried that there's so much else going on in my input image that applying the projection + thresholding equation isn't going to work. It almost seems like I need to sub-image my input many times and try over and over again to get the right projection (almost like multi-template matching). Or maybe I can detect blobs first, create sub-images from the blobs, then try the projection.

Because I have multiple angles of the model I suppose that handles rotational invariance, but what about scale invariance? Seems like whatever subimage I get from my input, I have to try at different scales. Or I'm not understanding something fundamental about how to use PCA.

Thank you for any help; for reference I would like to implement this in OpenCV (Java-based).
Principle Component Analysis by itself is not going to provide rotational or scale invariance. You still have to provide templates for rotational and scaled models for image/template matching. What PCA does do is reduce the dimensionality of the search space  by converting a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components.

Now since number of principle components &lt;= number of components, you save on the search space.

What the lecture tries to tell you is with the eigenvectors, you get an efficient representation of the scales of the template which are easier to manipulate(a single scale factor) then manipulating the entire image everytime.