Draft from under baseboard - pull back carpet or remove baseboard to caulk?
Attempting to put caulking to the wall under the base board with the carpet installed is very likely going to result in caulking spread out onto the carpet - also if you ever have to pull it temporarily - the parts embedded into the caulk will separate from the rest of the carpet.  Unfortunately, placing the caulking under the baseboard with the carpet removed also presents a problem; namely dirty, sharp, and possibly rusty pointy carpet grabbing spike heads - ready to puncture your fingers.  I would recommend using caulking backer rod.  This is a flexible solid foam rope which could be pushed into the gap conforming to the space and sealing off any air penetration.

A more permanent solution would require crawling; 
If this is an interior wall and access to crawlspace or basement is available  I would suggest inspecting the floor for penetrations which could be sealed.  If it is an exterior wall check for penetrations through the wall plate (bottom and top) at the perimeter and any exterior penetrations for outlets.  horizontal siding is notorious for allowing air to enter walls; without a well sealed vapor barrier under the siding, there is little that can be done without opening the walls.