Itchy ears can sometimes be a sign of an ear infection. Bacteria and viruses cause them, usually when you have a cold, the flu, or allergies. One kind, swimmer's ear, can happen when water stays in your ear after you swim. Too much moisture wears away your ear canal's natural layer of defense against germs.