Can I replace my bicycle's dynamo with batteries?
Yes, a battery holder and switch from radio shack would accomplish this. Most dynamos are 6vac and can be replaced with 4xAAs or the 5vdc from a USB pack. You can ignore the issue about polarity as the dyno lights that I've worked are either incandescent and don't care or are LED but have built in blocking diodes and don't care.

Whether you would want to do this is another issue. I've gone the opposite direction and put a dynohub in my commuting bike.

Note: if you are handy with a soldering iron and multimeter, you can remove the diodes/rectifier (and likely super cap) from the LED lights and have them run in straight DC. You'd gain brightness especially when your batteries are running low because you would no longer have the voltage drop of the diodes or rectifiers, but you would need to check polarity and make sure you aren't drawing too much current.