Should I repair small gaps between brick sill and window frame?
If the brick veneer was done well there's a "drain plane" behind it which sheds rainwater without allowing it to damage the structure. If not, this is a serious concern that should be addressed. 

If you wish to fill the gaps, do it with a material that best matches from an aesthetic standpoint. I'd consider a mortar-colored exterior grade caulk (ideally urethane), applied with as small a bead as possible. You may want to pre-fill the larger voids with foam or another insoluble backer.