As one branch of stereo matching, video stereo matching becomes more and more significant in computer vision applications. The conventional stereo matching methods for static images would cause flicker-frames and worse matching results. We propose a joint motion-based square step (JMSS) method for stereo video matching. The motion vector is introduced as one component in the support region building for the raw cost aggregation. Then we aggregate the raw cost along two directions in the support region. Finally, the winner-take-all strategy determines the best disparity under our hypothesis. Experimental results show that the JMSS method not only outperforms other state-of-the-art stereo matching methods on test sequences with abundant movements, but also performs well in some real-world scenes with fixed and moving stereo cameras, respectively, in particular under some extreme conditions of real stereo visions. Additionally, the proposed JMSS method can be implemented in real time, which is superior to other state-of-the-art methods. The time efficiency is also a very important consideration in our algorithm design. (C) 2015 Society of Photo-Optical Instrumentation Engineers (SPIE)