How can I remove a prop that is used in an action but not needed in another? Sorry if this is a newbie question, or maybe I'm not using the right search terms. 

I have different action strips for a character, and I'm using props in the first action strip, that are no longer needed in subsequent actions. However I can't seem to remove them without affecting the first strip too. 

Is there something obvious I'm missing?

Thanks in advance!
Found something even better! Mentalist gave me much better search terms and I've now found the best answer to the solution. I didn't even know the eye (visibility) icon in the Outliner could be keyframed before =P 

Thanks all for the help!

Animate the Viewport Visibility of an Object within the Timeline?