1. A voice retrieval device comprising: a memory; and a processor coupled to the memory and configured to: set detection criteria to detect a retrieval word, based on a characteristic of the retrieval word, such that the higher the detection accuracy of the retrieval word or the lower the pronunciation difficulty of the retrieval word or the lower the appearance probability of the retrieval word, the less number of sections to be selected, as candidate sections, from voice data including a plurality of sections obtained by dividing the voice data into a plurality of frames, the voice data being recorded using a microphone; select part of the plurality of sections as the candidate sections which possibly include the retrieval word by performing first voice retrieval processing on the voice data according to the detection criteria, the first voice retrieval processing including calculating a matching score using the detection criteria for each of the plurality of sections included in the voice data, the matching score indicating a possibility of the retrieval word being included in each of the plurality of sections, according to the first voice retrieval processing, and detecting sections having the matching score that satisfies the detection criteria as the candidate sections; detect a section including the retrieval word by performing second voice retrieval processing using the detection criteria on each of the selected candidate sections, the second voice retrieval processing being different from the first voice retrieval processing; and output the detected section which includes the retrieval word.