These small red bumps that develop on the back, chest, shoulders, and buttocks are caused by inflammation and irritation of the hair follicles and sometimes related to bacteria and yeast on the skin, says Shainhouse.