What is this flicker I get while tracking a camera?
You can clearly see the tracker shifting on the video posted on the question, and it happens at the moment where the reconstruction fails.
Successful reconstruction can only be achieved with accurate tracking
You should aim to have a solve error of less than 0.3 pixels.
There are plenty of tools to help you get more accurate tracking.
Read:
Solve error high with good track?
Motion Tracking. What do the graphs represent and how are they useful?
Does the motion tracker follow a point if it changes colour throughout the video?

For a comprehensive post on motion tracking tips read
How can I get better results when doing camera motion tracking?