How to use a driver to pan and zoom an image
Either you should be using a driver OR you should be using keyframes - not both since one will override the other. It sounds like you should actually be using keyframes - so right-click and Remove Driver.
Using keyframes the field should be yellow/orange/green to indicate the keyframe. Move to your frame, set the value, then hit ‘i’ to insert keyframe. No need for driver.