Soldering capacitors onto a board that is spinning
Okay, so the forces are too great. You have several (not mutually exclusive options): 


Decrease the mass of the capacitor. 
Improve the mounting. 
Find a part with more sturdy terminations. 


Perhaps you could substitute a (probably smaller) 10uF ceramic for the 4.7uF tantalum. 

Mounting can be improved by using epoxy (but it's not easily reversible)

Maybe a leaded (through-hole) part would be sturdier.