How do I change which half of a receptacle is controlled by a wall switch?
Yes, that's exactly what you should do.

As a safety measure, while you've got the wiring exposed, double check that the hot (black and red) wires are connected to the brass-colored screw on the receptacle, the neutral (white) is connected to the silver-colored screw, and the ground is connected to the green screw.