1. A speech retrieval apparatus comprising: a segment detection unit configured to detect one or more coinciding segments for speech data by comparing a character string of a recognition result of word speech recognition and a character string of a keyword, the keyword being designated by the character string and a phoneme string or a syllable string stored in a non-transitory computer readable storage medium; an evaluation value calculation unit configured to calculate an evaluation value of each of the one or more coinciding segments using the phoneme string or the syllable string of the keyword to evaluate a phoneme string or a syllable string recognized in each of the one or more coinciding segments and that is a recognition result of phoneme speech recognition, wherein the phoneme string or the syllable string associated with each of the one or more coinciding segments is a phoneme string or a syllable string associated with a segment in which a start and an end of the segment is expanded by a predetermined time; and a segment output unit configured to output a segment in which the calculated evaluation value exceeds a predetermined threshold.