1 Ingrown hairs are usually caused by shaving, waxing, or tweezing hair. 2  Ingrown hairs can cause pain and itching, and they can become infected.  Most ingrown hairs will improve without treatment, though creams or antibiotics may be necessary.