Why don't capacitor's plates stick together?
As you say, capacitor plates experience attraction due to the opposite charge. However this typically cannot be seen from the outside. The "plates" press against the insulating dielectric that separates them.

In rare situations one can hear the mechanical effects of the rapid release of the charge from the capacitor: the capacitor makes an audible "clack" when the charge is released instantly - i.e. through an almost short circuit.

This is a common occurrence with for example high-power pulsed laser power supplies: Such a power supply stores high-voltage energy in a bank of large capacitors for a period of e.g. 1/10th of a second. All the stored energy is then released in about one billionth of a second. This is accompanied with a fairly noisy "clack" that originates from the release of mechanical stress in the storage capacitors. - With large laser systems the discharge noise is so loud that ear-protection has to be worn in the vicinity of the power supply.