Power Supply Design 240VAC to 12VDC, what value smoothing capacitor do I need?
Update: I checked that link and it seems you want to use a 3 VA transformer to make a 60 VA supply !? You'll need a transformer of 75-80 VA.

First of all, a voltage regulator at 5A produces a lot of unnecessary heat.

Sticking to your design, I'll assume the voltage regulator has a drop of maximum 2V accross it. So it must be powered all the time with a minimum voltage of 14V.

So you have that transformer of 12 V RMS AC. After bridge rectifier you get \$12 \cdot \sqrt{2} - 2 \cdot 0.7 = 15.57V_{p-p}\$. This voltage should never drop below 14V. This means a maximum ripple of 15.57-14 = 1.57V. For even better reliability I'll approximate this to 1.5V.

Now the capacitor value is \$C = {I \over {2fV_r}} = {5 \over {2 \cdot 50 \cdot 1.5}} = 0.033 F = 33000 uF\$.

This high value makes your transformer unusable for this design. Now that you know how to calculate this, choose a transformer with higher output voltage so that the filter capacitor will have a smaller value.