Impact of capacitor on half-wave ac to dc rectifier
You bought a 25VDC capacitor. There should be close to 40VDC across it, so it's probably leaking like crazy. 

You need at least a 50V-rated capacitor for this. The peak voltage (ignoring the diode drop) is \$\sqrt{2}\approx 1.414\$ times the RMS voltage - what your meter purports to read- so about 40V for 28VAC sine wave in. 

But many regulator circuits will either be marginal or into prohibited territory with that much input voltage. Linear regulators, even if they can handle 40V in, will likely get too hot- they're wasting more than eight times as much power as the load uses. Really, you'd most likely be better off to buy a wall-plug adapter that provides a regulated 5VDC.