Where to Caulk Around Windows on Exterior of Home? Is this a Drip Edge?
That looks like a metal L-bar which is holding up the bricks.
Overall I would not caulk that because water will have nowhere to escape. Here is how water will likely travel:

Don't close off its escape route or else you're asking for trouble.
Overall I do not see an accessible place to caulk which would prevent drafts.
I would schedule a consultation with some window or energy companies to assess how to best seal your windows. They will likely point out problem areas and explain how they would fix them.
If you do end up caulking things then make sure to use exterior paintable silicone and not that cheap acrylic stuff.
The best bang for your buck is going to be to take off the interior trim and insulate the space between the window and rough-in frame with expanding foam. As you re-install the trim you should caulk with a high quality paintable silicone. This should fix draft issues. If you find that you also have moisture or rot issues then that is a completely different ball-game.

Other sources of draft/coldness:

Bad wool pile weather stripping
Window sash is not straight
Double pane seal is broken and argon insulating gas has escaped