Connecting LEDs to a 9V battery
Yes, you need a series resistor. The reason is that the LED voltage isn't exactly 4.5V, and it also varies with current. Having a series resistor is the only way to be sure you supply the right amount of current. You need some voltage drop across the resistor, however, so putting two LEDs in series is out. You need 4.5V across the resistor for 20mA so  


  \$R = \dfrac{9V - 4.5V}{20mA} = 220\Omega\$


A 9V battery can't supply very much current, but without series resistor the current may still be too much for the LEDs. I'm currently working with white LEDs with a typical current of 20mA, which have only 25mA as maximum.  

edit (re your comment)
In theory you could do without resistor, but the LEDs are never exactly 4.5V and your battery is never exactly 9V. If the battery's voltage decreases to 8.9V your LEDs will light (much) dimmer, or (much) brighter if the LEDs' voltage is 4.4V, since the current will be much higher, and maybe become too high.