Why isn't there an electrical potential?
They come from the mains, but they're isolated from it by a transformer. You're correct about the complete circuit, because that the condition to have a potential difference (just say voltage difference, that's also OK).

If you would take a transformer and measure the resistance between a contact on the primary and a contact on the secondary you'll see that the resistance is very high, almost infinite. There's no closed circuit. If you would measure the voltage between the pins of power supply A you would have a closed circuit: from one pin through the multimeter's resistance back to the other pin.  

A closed circuit is the condition.  

edit
However! Note that this is for DC. AC signals can pass though transformers and capacitors, which also would block DC.