1. A speech recognition device for recognizing an input speech of a word sequence based on a plurality of speech models which are modeled so that a possibility that a specified word or words are contained in the input speech is output as an occurrence probability based on a speech parameter, the speech recognition device comprising: a first speech model network for specifying a linking relationship among a plurality of first speech model groups, in which the speech models are grouped to include different specified words, the word sequence being a segment of continuous speech; a garbage model connected to the first speech model network for increasing the occurrence probability when a speech parameter is given corresponding to speech other than the specified words, which can be recognized by the speech models of the first speech model network; a second speech model network for specifying a second speech model group, in which the speech models are grouped to have common linking relationship to the speech models of the first speech model network; a speech model storage unit for storing the first speech model network, the garbage model, and the second speech model network; a data storage unit; a speech parameter extraction unit for extracting a speech parameter for the input speech; a speech parameter storage unit for storing the extracted speech parameter; and a matching unit for recognizing speech based on the speech models of the first speech model network and the second speech model network, the garbage model, and the speech parameter stored by the speech parameter storage unit, the matching unit comprising: a loading unit for loading the speech models of the first speech model network, and the garbage model into the data storage unit; a first occurrence probability accumulating unit for accumulating the occurrence probabilities by giving the speech parameter stored in the speech parameter storage unit to the first speech model network and the garbage model loaded into the data storage unit; a speech model network switching unit for selecting the second speech model network based on the accumulated occurrence probability of the speech models of the first speech model network when an occurrence probability output from the garbage model exceeds a predetermined value, and then loading the speech model of the selected second speech model network into the data storage unit; a readout position rewinding unit for rewinding a readout position of the speech parameter in the speech parameter storage unit by a predetermined number; and a second occurrence probability accumulating unit for reading out the speech parameter from the rewound readout position and accumulating the occurrence probability by giving the read out speech parameter to the loaded speech models of the selected second speech model network.