How do I check my upstairs bedroom for insulation in the ceiling?
If the house is old enough to have a plaster ceiling, there is a terrific chance there is no insulation. 

You would be wise to cut an access somewhere and get rent a blowing machine for insulation. get about 16" up there. Taper it to the roof rafters so it does not touch the roof. It will allow the roof to get too hot in those places and deteriorate the roof prematurely. 2' of airspace over the insulation is recommended.

Be mindful of any added can (recessed) lights that were put in later, they need to be IC (insulation contact) rated or keep the insulation off them too.