Can I "Set Active Object as Camera" without also forcing the 3D view to change position? (Blender 2.82a)
There's no need to look through a camera to make it active.
Simply click the green camera icon and go View > Align View > Align Active camera to view.

The shortcut Ctrl + Numpad 0 automatically looks through active camera, which isn't what you want here.