peltier cooler heatsink
Power is a difficult word here. A heatsink will not directly change the amount of thermal energy the peltier device can transport (from hot to cold side). It will, however, enhance the distribution of the thermal energy (i.e. cooling) in the fridge. Therefore, a controller (if the fridge has one) will be able to more often use the most efficient operating point of the peltier device and thus reduce the time to (re)reach the desired temperature. But wether you actually notice an effect depends a) on the isolation of the fridge and b) on wether temperature distribution inside the fridge matters to you.

My recommendation: Try it. Nothing bad can happen and you might see an effect.