1. A speech synthesis system, comprising: a sound signal acquirer configured to acquire a sound signal including a speech signal vocalized by a speaker and a noise signal; a speech recognizer configured to recognize speech vocalized by the speaker in the speech signal included in the sound signal; a first spectrum generator configured to generate a spectrum of the sound signal acquired by the sound signal acquirer as a first spectrum; a second spectrum generator configured to generate a second spectrum, based on features of localized phonemes recognized by the speech recognizer so that the second spectrum does not contain a spectrum of the noise signal; a modified spectrum generator configured to generate a modified spectrum by multiplying the first spectrum by the second spectrum; and an outputter configured to output a synthesized speech signal of the vocalized speech based on the modified spectrum.