Dimensions in Eagle Cad
Use the Group tool and move both the hole and the measurement.

Hint: You can use the move command and then select each hole and measurement by holding the Shift key. (Deselect with Control)

Edit:





I hope I understood it right.
You have to select one side of the measurement only and the hole of course.