What should physics majors work on during semester breaks?
The summer after sophomore year should be late enough to start doing research. There are lots of questions / tutorials about how to start research; best to just talk to your professors and/or apply to research programs such as the REU. This is also a good time to start studying for your physics GRE.

For the summer after freshman year, it's probably too early to do the above (though it's worth asking your professors about research options). I recommend working on soft skills -- writing, people skills, presentation skills, and programming all come to mind.