High peak current supply circuit
You can place a capacitor in parallel with your power supply to provide additional current demanded by the load for short periods. However, the voltage output will drop when the current demand exceeds the power supply's capacity. If the "output voltage should not reduce from 12 volt", then you will need to tune the power supply to provide slightly above 12 V during the low-current periods.

The amount of the voltage sag can be determined from the basic capacitor equation

$$Q=CV$$

In this case, \$Q\$ is the amount of charge to be provided during the surge, given by \$I t\$, where \$I\$ is the excess current demand (5 A in your case), and \$t\$ is the duration of the surge demand (1 ms in your case). So you need 5 mC of charge to be supplied by the capacitor during the current surge. Then

$$\Delta{}V = \frac{I t}{C}$$

Or, if you first determine the allowed voltage sag (\$\Delta{}V\$), you can calculate the needed capacitance as

$$C = \frac{It}{\Delta{}V}$$

So if, say, you allow for 100 mV of voltage sag, you need to have a capacitance of 50 mF (millifarads, not microfarads). Be prepared for sticker shock when you price out what 50 mF of capacitance rated for 15-20 V working voltage will cost you. If you can accept a larger voltage sag, you can then use a proportionally lower capacitance value. 

But once you price out 50 mF capacitors, you might want to look again at regulators capable of sourcing the full 10 A out of the box.