What is the standard way to make a Peltier power supply with variable power?
For more information see both my answer and Olin's answer at How to drive a Peltier element. Note that while our two answers may appear to conflict somewhat in places they are both giving essentially identical advice.


My key message is that thermal cycling must be avoided as if it occurs it will physically  destroy the device in much less time than when driven properly. Drive should be either DC with a low enough  ripple component that it does not affect thermal cycling, or PWM at a high enough frequency that the PWM frame period is far smaller than the Peltier device's thermal time constant. 50 Hz or greater OWM frame rate is probably OK.
Olin's key message is that PWM is less efficient than DC because the Peltier is less efficient when driven to maximum level, and PWM uses maximum level or off. 


RC filtering PWM will produce lossy DC and reduce efficiency. LC filtering PWM can result in higher efficiency.

If efficiency is especially important a good and relatively low cost solution is to use a buck converter running at a high enough frequency to avoid thermal cycling.  

Using my favourite entry level smps IC, the MC34063, you can build a suitable buck converter for probably $1 to $2 US. Fig 11 / 11A / 11B in the MC34063 data sheet show circuits for step down. You would want to convert these to constant current output and select the desired current to suit.