Preprocessing to improve Harris corner tracking between video frames? In our unmanned aerial vehicle grayscale video image stabilization application, we're having difficulty finding the "good" Harris corners in frame N+1 selected from frame N. The source of the difficulty appears to be radical nonuniform pixel contrast changes between frames. Perhaps being airborne and using a slow frame rate (~3 fps) is the root cause of the pixel contrast shift.

We've tried various histogram equalization techniques to try to smooth out the pixel contrasts between frames in order to improve tracking "good" Harris corners in frame N+1. Results continue to be poor.

Does anyone have any suggestions on how to improve Harris corner tracking between video frames in a slow frame rate daytime aerial environment? Thank you kindly in advance.

Edit: 30 Jan 2012, added test case (not actual frame size) images

Summary Update: 8 Feb 2012. People suggest Harris corners are not so useful in grayscale video feature tracking. Answers below suggest and provide links to various alternatives. We are evaluating these alternatives and I will report results when we get to that point. Thank you, all, for your comments and answers.

Here's the previous frame N with 35 "good" 5x5 harris corners selected. The original frame is 8 bpp raw pixels.



Here's an excellent 5x5 Harris corner located at row 59 col 266:



The current frame N+1 with a few tracked 5x5 Harris corners, only one of which is valid:



The previous frame 5x5 Harris corner appearing in frame N+1 at r47 c145:



Note how the pixel intensities in the selected 5x5 have all changed in a nonuniform manner from the previous frame to the current frame. Contrast equalization techniques between frames do not help in detecting previous frame selected 5x5 pixels in the current frame. All suggestions welcome.
Can you try a different feature detector? FAST may be, erm, faster, and a higher frame rate will make matching easier (assuming your features are moving a lot between frames)



Looks like you are trying to use the grayscale region around the identified feature point to match from frame to frame.  This is likely to be poor, especially if there is lots of movement between frames.

You may get better performance by using what is often called a "descriptor" of the region around the feature point on which to match.  

Some possible descriptors are used within the SURF and SIFT algorithms - they also have techniques for identifying regions to track, but you've bypassed the requirement for that.  A simpler descriptor designed for use in stereo-vision matching is the Census transform although that may not work as well for matching from frame to frame, for much the same reason as your current method (which is also widely deployed in stereo-vision)

The book to read on this is Multiple View Geometry in Computer Vision.