Moving Crop in Video
Similar to Gyan's overlay answer, a slightly simpler version can use a basic -filter_script with the "swaprect" and "crop" filters.  Move the crop target to 0,0 and then crop the target area at 0,0.  The target has to be swapped to 0,0 as otherwise the other swapped rect can overlap.  This worked with ffmpeg version 3.4.6-0ubuntu0.18.04.1

Make sure the frame numbers are relative to the offset passed with "-ss".

An example targeting 1280x720 crops from a 4096x2160 source.  The source is 24fps and the tracking info is 8fps.

crop-filter-script:

swaprect=1280:720:0:0:1568:594:enable='between(n,0,26)',
swaprect=1280:720:0:0:1552:598:enable='between(n,27,29)',
swaprect=1280:720:0:0:1565:583:enable='between(n,30,32)',
swaprect=1280:720:0:0:1603:576:enable='between(n,33,35)',
crop=1280:720:0:0


ffmpeg -ss [start offset] -t [duration] -i input.mov -filter_script:v:0 crop-filter-script -acodec copy out.mov