Why does winter get colder after the solstice?
Primarily because of inertia.  This phenomenon is called seasonal lag.

It is true that the December solstice is the moment that the northern hemisphere gets the lowest total amount of insolation.  Conversely, the June solstice corresponds to the moment of maximum insolation.  The oceans, and to a lesser degree the land, absorb a lot of heat.  Due to the large heat capacity of water and the large amounts of water, the seasonal lag can be considerable.  In general, mid-latitude areas near large bodies of water have the largest seasonal lag, to the extent that the warmest month of the year in most of western Europe is August.

It is illustrated very well by this Warmest day of the Year map that NOAA NCDC (now NCEI) produced for the contiguous United States:



As expected, areas with a humid climate, such as southern Texas and nearby areas, tend to have the warmest day of the year in August.  The west coast is even more extreme, with some places having the warmest day in September.  Conversely, desert areas in the southwest (southern Arizona, New Mexico, western Texas) have the warmest day of the year in June.  For the Big Bend area in Texas, it's even in the first half of June, which must have entirely different reasons (probably related to dynamics) because a negative seasonal lag doesn't make much sense.
 There is a corresponding map for coldest day of the year:


(Source: NCEI)

The detailed structure is somewhat different, which has to due with local climate and circulation, but the overall trend related to humidity and proximity to ocean shows up in both.  We can also see the mountains, which I would guess to be a snow albedo feedback effect (ever notice how nights get colder when the ground is snow-covered?).  At NCEI you will also find similar maps for the rest of the USA.