Are the plastic covers where the plug is inserted remove able so a different color one can be used?
There are cover plates which cover the receptacle and not just the surrounding space. I don't know if there are problems with them or not. They do cost more than a standard cover plate. There are different shapes of receptacles and one might have to get a cover which fits the type of receptacle one has.

In any case one never has to "change the entire box". The box is not visible when a cover plate is in place. The box is fastened to framing in the wall. What one might have to change is the receptacle inside the box. The receptacle is screwed to the box and has the wires attached to it.

The usual procedure for changing the color exposed to the room is to change the receptacle and its cover plate together. This may be what you meant by changing the entire box. If so, one shuts off the breaker to that circuit, removes the cover plate, unscrews the two screws holding the receptacle in the box, pulls out the receptacle, detaches the wires from the receptacle (usually by loosening screws) and the receptacle is free. Installing the new receptacle is just the reverse. This is a simple and straightforward procedure for an experienced DIYer, but for someone who has never done it it has pitfalls.