3 Watt RGB LED off of 3 AA batteries
For 3 W your current will be 350 mA per LED (red, green, blue). That's 1 A total.



Figure 1. The Energizer E91 datasheet to get an idea of what you can expect from an AA cell.

Note that Energiser don't even show the data for a 1 A discharge. At 500 mA the capacity is about 1500 mAh. Note that the graph is showing a geometric progression and so at 1000 mA discharge you could expect &lt; 1000 mAh capacity. The batteries would last one hour at 1000 mA. They would also get hot.


  Is this correct or do I need more/less voltage? 


You need a little more than the maximum VF of the RGB LED. The datasheet  (which is what you should have linked to instead of the catalog page) shows that this is 3.6 V for the blue. 4.5 V is the next multiple of 1.5 V above that so 4.5 V is the correct choice.


  I am also worried about the batteries dying and slowly losing their voltage. Could this damage the LEDs?


No it will not damage the LEDs.


  What resistor would I use?


You need individual resistors in the red, green and blue.

\$ R = \frac {V_{batt} - V_F}{I} \$ where \$ V_F \$ is the forward voltage of the LED in question.