Algorithm for object tracking In a video sequence, I would like to track an object. Right now, my detection algorithm can correctly locate the object in approximately 10% of the frames, and incorrectly find something else in 5% of the frames. 

The incorrect detections are scattered around in the image, while my object movements are constrained by physics. 

I've tried implementing a simple tracker based on distance from last detection (frames and pixels), but since my object moves in a 3d world, the velocity can vary a lot.

I think there is a rather simple approach for tracking this, but I don't have a lot of prior tracking knowledge. Where would I start?

(I appologize if this is not for DSP, please redirect me in that case)
assuming you have a relative high frame rate video (the object is moving relatively slow between two consecutive images) some standard tracking algorithms can be applied.

Mean shift [Wikepedia - Mean shift]
Use some template of the object you are tracking to create a likelihood function for the object location and track the object.


if you need more advanced method that can be more robust, e.g. object occlusions you can use more advanced methods:

kalman filter [Wikipedia - Kalman filter]
In short kalman filter uses a dynamic model that is composed of
prediction step and update step to track an object in an unsertain
enviroment.


More advanced option you can consider, if you have couple of objects you want to track at the same time is particle filter