What's the relationship between volts and amperes in batteries?
First thing to say is that whenever using laser diodes (or LEDs for that matter) you need to limit the current into the device. You can't rely on the battery voltage for producing exactly 4.5V x 2 because if it produces 4.6V x 2 then you might find that the current into the laser diodes (2 in series) would go from 50mA to over 100mA or much higher leading to destruction of the laser diode.

Like LEDs, laser diodes have a typical forward volt drop (4.5V in your case) at the rated current (say 50mA) and some may be only 4.4V and if you had a bunch of these they may well have died and you'd be regretting spending the money so use a series resistor or an active current control mechanism.

Without seeing the data sheet I can imagine that full 5mW laser output is achieved with round about 50mA of current and that your battery arrangement has just enough internal series resistance to limit the current to safe levels.

If a pair of lasers in series took 50mA then 4 pairs would be taking 200mA and already a PP3 battery would start to reduce its output voltage and dim the lasers. Look at the discharge curve for an energizer 522 9V battery: -



With 200mA load after 2 hours the voltage will be at 4.8V. You could estimate that after 40 minutes it would be 7.6V and well below the 2 x 4.5V needed for the laser diode. You could estimate that each minute that passes the battery terminal voltage drops 35mV and after 2 hours this drop is 4.2V hence only 4.8V left at the terminals!!

In fact, if the laser diodes were exactly 4.5V then after a few minutes there would be not enough voltage to activate them. And this was a battery that had been used to drive a few LEDs for a few hours!

The battery pack made from several 1.5V batteries can generate much more current for a longer time period. All the same, if I was using a battery pack I'd make it 6V and have individual current limit resistors for lasers i.e. don't wire in series.

To get more out of the 9V battery (and you won't get a whole lot more) you could drive each one individually from the 9V with a current limit resistor. You need to find out what the forward current of the laser is. I can guess at 50mA and this would mean a dropper resistor would be: -

\$\dfrac{9V - 4.5V}{50mA} = 90 ohms\$