Move object relative to camera instead of moving camera relative to object
You can do that by copying and manipulating the F-Curves in the Graph Editor.
Essentially, you need to copy the f-curves relating to Location (you can do this with Ctrl+C when you have selected them) and paste them into the F-Curves for the object (using Ctrl+V) (you'll need to add a keyframe first to create the curves for the object).
Once you've copied the f-curves you can add a Generator modifier with a 1st order polynomial in the format y = c + mx, setting m to '-1' (so as to 'flip' the curve) and 'c' equal to an offset matching the difference between the coordinate of the Camera and your object at the start of the animation.
Once you've done this you should get the same (but opposite) motion between your camera and object - so all that's left is to remove the keyframes from the camera so that it remains static while the object moves in the opposite way to which the camera previously animated.