Will a transformer with no output overheat
No. There is the assumption that it is in good condition and not subject to invasion of moisture. Self-heating helps with this issue. A doorbell transformer with no load stays at a idle current state, consuming only a few watts off of a 120 VAC or 220 VAC line. This will keep it warm to the touch, but if you have icy-cold winters the metal case will become cool or cold to touch.
The only risk with a transformer with no access to disable is a sharp rise in line voltage due to lightning or a cut neutral wire, or pinched output wires. It would behoove you to find what circuit breaker disables this transformer and mark it, just in case.