Possible to determine a device's power requirements based on the batteries it takes?
Batteries can only supply a given amount of current. You could use an adapter that supplies the same voltage and exceeds the current capacity of the batteries to properly power the device.

I would recommend using a current meter to simply measure what the device uses: Get a 3X AA battery holder, and insert your meter between the batteries and the lamp. Measure with fresh batteries and you will have your current usage. An adapter that exceeds the value by at least 10% should suffice.

If your device has a DC power jack that isn't labeled, you will have to investigate as to whether the tip or ring is ground. Obviously any adapter you purchase could be modified to match a given polarity; many have plugs that can be attached either way. (I personally solder my own using adapters that come with bare wires.)

Make sure you get a regulated power supply. An unregulated one can supply more than 4.5 volts if not powering the load it was intended for.