Disable Tweak mode / hide screen space transform gizmos?
You can turn off the gizmos completely using the Overlays options at the top of the screen. I personally have them off because they don't serve much purpose once you're committed to the use of hotkeys to do all your transforms.