Tracking an object in a video with
I think I can come up with two suggestion to address this problem.


First (simple)


You can simply use HSV color space, and try to find right thresholds. It seems in HSV colour space, the car is pretty distinguishable from the background but the rail is the problem now.

I=imread ( your image );
J=rgb2hsv(I);
imshow(0.1&lt;J(:,:,2) &amp; J(:,:,2)&lt;0.2 &amp; 0.2&lt;J(:,:,1) &amp; J(:,:,1)&lt;0.54,[]);



Second (a bit more complex)


Since your background is presumably static, you can remove background and the remaining would be your car (and of course the rod). I suggest to use optical flow for this purpose.