'Voltage Fuse' or better
Power supply in this circuit is +12V.  Zener voltage and current rating of the PTC will depend on the specific power supply and load, of course.

In case of overvoltage, the Zener diode starts to conduct and clamps down the supply voltage.  At the same time, power supply current goes up.  Then either power supply becomes regulated, or PTC fuse opens.

For normal operation, this circuit should have margins.  The Zener voltage should be slightly higher than the nominal power supply voltage.  The max voltage, which load can tolerate, should be somewhat higher than the Zener voltage.

(I really like this circuit, because it protects against: overcurrent, overvoltage, reverse polarity.  And it does that with just 2 components.)

edit: related post about this type of overvoltage protection.

crowbar circuits

Unlike the clamp circuit above, the crowbar circuit pulls the voltage below the trigger level.

overvoltage and overcurrent protection ICs

There are also overvoltage and overcurrent protection ICs.  These are typically controller chips, which work with external FET switches.  Some examples can be found here and here.  

The ICs called hot swap controllers sometimes have overvoltage and overcurrent protection functions too.