Understanding capacitors
Your theory is correct, but not complete.

When you start discharging capacitor - voltage will drop. When you draw 50% of energy from capacitor - capacitor voltage will drop by 50% by 30%. With power supply connected in parallel - capacitor voltage will not drop so deep, because current will just start flowing from supply, and you will be unable to get energy from capacitor.

Edit: image removed.

More useful information: link

Many power supplies have rating determined by temperature.

If you have power supply with transformer - you can overload it by 20% with no problems at all if you "let it rest" later.

If you have supply based on voltage converter (transformerless) this is more complicated - you may have problems with voltage stability, voltage drops, temperature etc. In general - you shall not overload transformerless power supply, even for a short time.

And one more thing about big capacitors - they are able to draw very large currents when they are charged. Supercapacitors should be charged from current limiter.