Converting Christmas Village from DC to AC
The simple answer for the houses that run from 3* 1.5V cells is to simply use an off-the-shelf cell phone charger.  These are nominal 5 Vdc output.  Note that most alkaline cells start off at about 1.6 Vdc but decay quickly down to the nominal 1.5V value.

If you are concerned that 5V is a tad bit too high, simply add a standard silicon diode in series with either lead, making sure to observe polarity.  This will drop the voltage down to near 4.3 Vdc or so.

The street lamps are a tiny bit more work - just use more silicon diodes in series with the power supply leads that feed those lamps.  Two diodes will drop your 5V power supply down to near 3.3 Vdc.

I think highly of Genuine Samsung cell phone chargers: nominal 5 Vdc at 2 Amps.  You even may have some hanging around from old phones.  If not, they are readily available from eBay and other sources.  Just be sure to purchase Genuine chargers - some of the Asian-made knockoffs are complete trash.