Tantalum capacitor burning out
Tantalum capacitors fail if you install them backwards. No surprise here.

They may also fail due to high surge current, e.g. if the capacitor is initially empty and then connected to a power supply with high peak current capability.

When connected to a power supply a high initial current will be present at the capacitor (can be in the order of 10 Ampere and more). This is due to the very low internal series resistance of the tantalums.

Albeit this is a short event it causes heat and stress within the capacitor and may result in a failure. To find out if this is the culpit you should - as a test - connect the tantalum with an additional series resistor of lets say 1 Ohm and see if this helps. If it does you have to limit the initial inrush current of your device. 

Note that you may see such high current spikes even if your power supply is only rated at 500mA or so. One or multiple large and low ESR output capacitors are all what is needed.