Voltage increase with capacitor... what's the "true" voltage?
Figure 1. Bridge rectifier and smoothing capacitor. Image uncredited on EE.SE.
Without the smoothing capacitor you get the dotted-line full-wave rectified waveform. You are reading the average value of this, 12.3 V.
The smoothing capacitor, when added in, is charged to the peak value of the voltage and maintains it during the dips in the rectified voltage. The average voltage will be close to the peak voltage, 20 V.

When I hook up a 12v load (light bulb) it goes back down to around 13v.

The capacitor's value isn't high enough to maintain the voltage at 20 V with such a large load as your lamp. It does hold it up 0.7 V higher than without the capacitor.

Why is this voltage increase happening with the cap?

Explained above.

If load decreases the voltage in the circut, how do I know I'm not sending too big a voltage to my lamp?

You use the meter to check this. With 13 V you are running a little &quot;hot&quot;.
Note that incandescent lamps are tolerant of peaking voltages due to the thermal characteristics of the filament. Electronic devices may be destroyed if the peak voltage exceeds the device's rated voltage.

Note: 'V' for volt, 'A' for ampere, etc. SI units named after a person have their symbols capitalised but are lowercase when spelled out.