Can a capacitor be used for overvoltage protection?
Relying on a capacitor to fail in an overvoltage condition in order to protect some other equipment is a bad design practice. Capacitors may fail open, short, or somewhere in between; unpredictability makes for poor protection. And I've personally seen overvoltaged caps emit flames.

You should spec all your components such that its voltage rating is at least the highest voltage you expect them to ever see, plus some safety margin. If its unexpected voltage spikes you're concerned about, you can use more capacitance on a DC link to limit the voltage rise, or MOVs on an AC line connection.