This paper proposes an approach for a robust tracking method to the objects intersection with appearances similar to a target object. The target is image sequences taken by a moving camera in this paper. Tracking methods using color information tend to track mistakenly a background region or an object with color similar to the target object since the proposed method is based on the particle filter. The method constructs the probabilistic background model by the histogram of the optical flow and defines the likelihood function so that the likelihood in the region of the target object may become large. This leads to increasing the accuracy of tracking. The probabilistic background model is made by the density forests. It can infer a probabilistic density fast. The proposed method can process faster than the authors' previous approach by introducing the density forests. Results are demonstrated by experiments using the real videos of outdoor scenes.