Powering off USB - question about grounding
When you place your multimeter leads on two pins, it creates a circuit that allows your multimeter to measure the voltage.  If you don't connect both leads of the multimeter to form a circuit, the multimeter will not measure the voltage, even though the voltage is still present.

If you are using the USB socket for power, then you need to connect the ground of the socket to the ground of your circuit.  At a simple level and without knowing more about your circuit, as long as you connect ground of the USB socket to your circuit ground somewhere you should be fine.