Change camera start tracking time / change values
You need to animate the influence value on the "Track To" constraint.

If the value is 0 the camera will not track the object. 

When the value is 1 the camera will track the selected object.

To animate the value you need to create keyframes, just place your cusor over the Influence value, right click and select Insert Keyframe (or simply press I while the cursor is over the value).



Keep in mind that when the influence is 1 the tracked object's origin will be in the center of the frame, so going from 0 to 1, or from 1 to 0 might change placement of the object within the frame. For smoother transitions make the value change over the course of several frames.