How can I make it so that objects only move with widget / text entry?
In order to not to grab / move objects with RMB, you should change tweak behaviour.

1. Increase the tweak threshold 

To do that increase the Tweak Threshold in the User Preferences > Input:



Set it to something like 100 px. This means the mouse will need to move being right-clicked at least 100px in order this event to happen.



2. Turn off the tweak event mapping

In the User Preferences search for "tweak". In the results in 3D View section find and turn off Translate item with type of event mapping Tweak (or change keybinding for it):