How do I supply a short but high power pulse (higher voltage and current) from a low power source?
What you are describing is commonly used for flash bulbs where you need a lot of current, but only for a short period of time. You limit the rate at which the capacitors can be charged as to not pull too much current from your source, but then the capacitors are able to provide a lot of current for a short time. This is the reason you have to wait a bit between taking pictures with a flash.

The concept still applies when you are wanting to apply a voltage greater than your source except you need a way to step up the voltage. For this I would recommend a DC-DC boost converter. The boost converter is also recommended in this question for high current applications. You can learn more about the boost converter in any of these questions.