Should the tiny gap between exterior window trim and the window housing be caulked?
Yes for sure.   The caulk is rather important there more than anywhere else because water will run down those channels and rot out the window framing and window itself (rust if metal).   

I suggest ge5020 concrete and masonry silicone ii and to do this window in clear or black.  I would not match trim color as something that light will show dirt more and you will end up with brown or grey very soon.