How to correctly LOCK motion tracking points so they don't move after reloading movie clip?
As I remember; from the introduction of the tracker, it was always suggested to export a movie to an image sequence. This is to prevent decoding changes/errors to frames when scrubbing through, skipping and reversing direction, as all these conditions can affect the decoding process.
The symptoms of the problem you describe sound exactly like this problem.
I would suggest you convert your movie into individual frames, and then work on that image sequence.