Reconstructing camera panning movement in 3D view using track data for Loc only
What happens, when you press the Link Empty to Track buttons is, that an empty gets added with a Follow Track constraint. We can just add this constraint manually to another camera.


Move the active camera to the position of your object.
Duplicate the camera and rename it to Moving Camera.
 RMB select the Moving Camera and &#8679; Shift RMB select the original Camera and direct parent the Moving Camera to the Camera by pressing &#9096; Ctrl&#8679; ShiftP > Make Parent without Inverse. The G grab the Moving Camera and move it to the approximate where the object would be in the original Cameras view.
Add the Follow Track constraint to the Moving Camera and set the Camera as the Object and Camera of the constraint. Choose the correct movieclip and track as well.

Then press Constraint To Fcurve on the constraint. This will bake the constraints movement to fcurves, which we can examine in the graph editor.
In the graph editor flip the keys vertically by pressing &#8679; ShiftM > By Values over Value=0.