Repair a crack in a brick windowsill
If the cracked bricks are loose (as in when you wiggle them they move) they should be removed and new brick installed (or clean the old ones of mortar).   If the cracked bricks are sound and secure the gaps should be filled ,as you noted, to keep water from further damaging the sill.  There are many colors of concrete-type caulking, so if desired the gaps can be filled with a red color and the joint with a grey color. Use a urethane caulk for greater longevity.    A two part epoxy would work just as well as a caulk but it would be difficult to color match (if that is important). Epoxies can be purchased at a local hardware store or a contractors warehouse.