Amplify current from a 4-20 mA temperature controller to drive a Peltier module
I think you are at the answer already. You need to convert your 4-20 mA analog signal to a PWM signal to run the Peltier. The Peltier is not affected by running PWM providing you keep the frame rate high, anything over 1 KHz should be just fine. 

There are professional solutions for 20 mA to PWM or of course you could convert to voltage and go from voltage to PWM. 

While the battle between those for/against PWM of the peltier modules rages, I tend to believe those professionals such as TE, who almost exclusively produce PWM controllers. This paper shows tests of the Peltier modules at differing PWM frequencies. Notice particularly that in these tests you CAN see changes over the control unit with time, but the effect is below 2% in 2500 hours of operation.
Also this rather interesting Masters paper shows one reason to use DC control for low noise (which is an exception to most requirements).

TE have a range of controllers (all PWM drive) with lots of technical info on the site