5v Battery - smaller is better
Use a single Li-ion cell with a boost converter to 5V. Microchip makes the MCP1640 boost converter chip, and other manufacturers make similar devices. Here is an app note describing the technique. One of the examples delivers 5V at 200 mA. The circuit could be made very small, to go with a coin cell. It has a big advantage over two cells and an LDO - it will start with a voltage as low as 0.65V, and still work with the cell discharged to 0.35V, giving a very long cell life.