A Dr. Jonathan B. Levine, DMD, Dentist, answered. A lack of different vitamins can cause several oral health problems: 1  Vitamin B2 (riboflavin) deficiency: Shiny red lips, sore tongue, cracks and sores at the corner of the mouth.2  Vitamin B3 (niacin) deficiency: Red and/or swollen tip of tongue with dry, smooth edges, sores at mouth edges, general mouth pain. Vitamin B12 deficiency: Bad breath, sores at edge of mouth, bright red tongue that may have fissures, loss of taste, dry mouth, numbness and bleeding. 2  Vitamin C or A deficiency: Bleeding gums, lowered immune response, infections (like yeast infection) in mouth and impaired taste.