Preprocessing to improve Harris corner tracking between video frames?
Can you try a different feature detector? FAST may be, erm, faster, and a higher frame rate will make matching easier (assuming your features are moving a lot between frames)



Looks like you are trying to use the grayscale region around the identified feature point to match from frame to frame.  This is likely to be poor, especially if there is lots of movement between frames.

You may get better performance by using what is often called a "descriptor" of the region around the feature point on which to match.  

Some possible descriptors are used within the SURF and SIFT algorithms - they also have techniques for identifying regions to track, but you've bypassed the requirement for that.  A simpler descriptor designed for use in stereo-vision matching is the Census transform although that may not work as well for matching from frame to frame, for much the same reason as your current method (which is also widely deployed in stereo-vision)

The book to read on this is Multiple View Geometry in Computer Vision.