This paper proposes a novel high-speed visual target tracking system based on mixed rotation invariant description (MRID) and skipping searching method. MRID is a novel rotation invariant description of texture and edge information by annular histograms and dominant direction. It overcomes rotation variant and large computation issues in conventional LBP-HOG feature description. The skipping searching method used in tracking can remarkably decrease the computation time by avoiding repeated searching operations. The proposed tracking system contains an image sensor, a hierarchical vision processor and an actuator with 2 dimensions of freedom (DOF). The vision processor integrates processors with pixel-and row-level parallelism to speed up the tracking algorithm. Experiment results show that the proposed system can achieve over 1000-fps processing speed of the tracking algorithm under 750 x 480 resolution image.