Is it safe to keep on using AC adapter with a damaged output capacitor?
Most likely the popped capacitor is causing stability issues (output going up and down as you described).

It should be replaced with a part with comparable voltage, ripple current, ESR and life ratings. Failure to choose an appropriate cap will likely result in the cap failing prematurely.

A properly protected power supply should not be damaged from a blown output capacitor. I wouldn't recommend operating it until the cap is replaced.

It seems odd to me that the power supply is capable of self-damage. With 25V caps the max output ought to be below 20V. 

I would use a sharp utility knife to cut the part from the glue, then desolder it and replace it. Make sure you note the polarity.