If you get a sore, white mouth or tongue while taking, or soon after stopping Zithromax, tell your doctor. Also tell your doctor if you get vaginal itching or discharge. This may mean you have a yeast infection called thrush. Sometimes the use of Zithromax allows yeast to grow and the above symptoms to occur.