Capacitor as a ripple filter in the rectifier circuit - difference in charging and discharging time
The time constant for charging and discharging is same. How can charging for a small amount allows it to discharge for such a long time.

No, they are not the same. The capacitor charges from the rectified voltage source, only through the voltage source internal resistance and the diodes. When the input voltage drops below the capacitor voltage, the diode opens and the capacitor discharges through the load resistance, which is much higher (so the current is much smaller).
As an example, check this simple half-wave rectifier:

R1 represents the voltage source internal resistance and R2 is the load (the circuit powered by the rectified voltage).

After the transient and with a fixed load, the charge lost by the capacitor while diode is opened equals the one quickly recovered when the diode conducts, hence the higher peak current.