Calculate resistor for red Led in series
Red LEDs usually have a 1.7V drop.  If the LEDs are wired in series, they will drop about 5.1V.  9V - 5.1V leaves about 3.9V across the current limit resistor.  

I'm going to further assume that you want to have the LEDs run at 20 mA max.  So: 3.9V / 0.02 Amps = 195 Ohms.  The closest standard (E12) resistors are 180 or 220 Ohms.  I'd choose 220 Ohms.

Now let's see what happens as the battery dies.  A standard Alkaline battery is considered to be dead when its' terminal voltage drops to about 1V under load.  A 9V battery contains 6 cells.  6 * 1V = 6V.  (6V - 5.1V) / 220 Ohms ~= 4 mA.  The LEDs will be lit but dim.