Can't motion track backwards
Read: 

How can I get better results when doing camera motion tracking?

and 

Some Data Failed to Reconstruct, Motion Tracking

If there is motion blur or your footage is blurry, it is very unlikely that you will get accurate tracking regardless if you are tracking backwards or forward. 

Things you can try:


Make the search area for the tracker larger



(if the search boundbox is not visible enable it on the marker display section)

 
Use a different tracking preset:


Prefetch the clip (to prevent errors when decompressing the video frames)
Set the Motion model to Affine or Perspective
Use Match: Previous Frame instead of keyframe.
Enable Prepass and Normalize.




Note that there is a chance that  you might encounter errors tracking backwards if your video is encoded using Long Group of Pictures (GOP) encoding and not intra-frame encoding (look that up on your favorite search engine). Blender has issues reading backwards or dealing with video encoded with variable frame rates. In that case you would need to re-encode the video before tracking, or convert it to an image sequence before loading it to blender.

Read: 

https://blender.stackexchange.com/a/8635/1853

and 

Why is tracking getting less accurate after going past cached frames?