Why is it colder in the mountains than at sea level?
The traditional answer basically comes down to the physics concept of adiabatic cooling, a description of which is:

There is less pressure as you go up in the atmosphere (basically due to less air weighing down)
Air takes up more volume at lower pressures
Since there's nothing else to supply the energy needed to expand, the air employs the energy that was in the temperature (which is the amount of air molecule movement).  The result is that air higher up is cooler by about 10 C per km (for dry air... moisture releases heat, reducing the cooling to about 6 C per km)

But, in summary, air is cooler higher up in the atmosphere.
And that is certainly the dynamics explaining why air temperature decreases as you go up throughout the troposphere (the only layer of the atmosphere we generally encounter in our lives... other things do come into play in other layers... such as ozone collecting and rereleasing energy from the sun in the stratosphere, actually causing warming with height there).
However, it seems that this typical explanation actually falls short.  Because if you pay attention to the wording of that &quot;because air higher in the atmosphere is cooler&quot; phrase, it doesn't actually answer the question.
Because all locations you're considering are surface locations, whether they're at sea level or in the mountains.
And the surface layer basically does all the absorbing of the troposphere's sunlight (the solar radiation absorbed by the troposphere is negligible).
And after all, mountainous places receive the same amount solar energy as lower elevation sites (in fact, technically a tiny bit more, that tiny tiny bit I mentioned that the troposphere takes out for sea-level locations).
Where am I going with this?
If the ground gets the same energy.  And if it's made of the same material and has the same color, it would therefore gain the same amount of temperature.  And then the air's heating comes up from the ground below.
So then, once again, why are mountains cooler?!?
Because of their EXPOSURE.
Air predominantly flows horizontally rather than vertically (basically consider this an effect of gravity and the resultant air density distribution).
And it's that horizontal wind that changes everything.  Because most sites on mountains are generally quite proximate (at least regionally) to places where winds will blow in air that was higher above the lower elevation ground surface, and thus colder.  The air on mountains, with limited protection (and because winds are higher aloft) is mixed strongly with the neighboring adiabatically cooler air that's sitting above the sea-level sites.
Just for more convincing this is vital, I invite you to look at these climatographs comparing two cities in the US.

What are these towns?
Red: Wendover, Utah (elevation 4,291', latitude 40.73° N)
Blue: Latrobe, Pennsylvania (elevation 997', 40.32° N)
Over a km difference in height.  So you'd expect Latrobe to be about 18°F warmer.
But it isn't.  Instead it's cooler (especially during summer afternoons, which average up around 10 degrees apart).
Why?  Because a lot of the intermountain west is actually quite a wide flat elevated basin\plateau.  So in the central part of this area, there isn't as much exposure to the colder air coming in from places with lower topography.
Rest assured, it was very hard to find examples like this.  You need a very wide flat elevated area, which is fairly uncommon.  Plus just the right place to compare to for emphasis.
There are other common factors that do enhance the cold predilection on mountains, including the lower moisture content at elevation, the often lighter land color of raised terrain, and the increased cloud coverage due to air rising orographically.
But the fundamental reason it is colder on mountains is adiabatic cooling and the exposure of raised locations to that colder air.