Peltier temperature change speed
A lot of things come into my mind:


where do you measure the temperature? Is your control target the peltier temperature, the heatsink temperature or something else (air temperature)? The farther away the target temperature is in the chain, the more thermal capacitance and resistance you have in the way, the slower the temperature change.
If you really want to control the peltier temperature, then place the temperature sensor on the peltier.
If you want to control the air temperature, then place the sensor there on the right spot, but don't be surprised, that it is going to be slower.
thermal capacitances: mass needs time to cool down, if you can reduce the mass of the heatsink or the thing you want to cool down, you should be faster
you can decrease thermal resistance by applying thermal paste or similar thermally conductive substances between the surfaces
if you measure air temperature, you might need to get the air moving (fan)
if the target temperature is close to the peltiers, by that I mean that the delay is low enough, you might be faster with an on/off control - be careful though, it might result in overshoots/undershoots
you could try tune the PID parameters, but you won't be faster than with the on/off control, so that's your theoretical maximum with this setup and power
if you want to control the air temperature, you might get lower thermal resistance between air and heatsink, if you increase the heatsink surface area
if nothing else works, you might have to apply more peltier elements