1. An HMM (Hidden Markov Model) learning device comprising: at least one processor; and a memory device storing instructions for configuring the at least one processor, wherein the at least one processor is configured by the instructions to: learn a state transition probability as the function of actions that an agent can execute, with learning with HMM performed based on actions that said agent has executed, and time series information made up of an observation signal observed as a result of said actions; and store learning results of said learning as internal model data including a state transition probability table and an observation probability table; said learning including calculating a frequency variable equivalent to frequency to be used for estimation calculation of an HMM state transition probability, and a frequency variable equivalent to frequency to be used for estimation calculation of an HMM observation probability; and said storing including storing at least said frequency variable corresponding to each of state transition probabilities, and said frequency variable corresponding to each of observation probabilities, of said state transition probability table; wherein said learning using said frequency variables further performs said learning to estimate said state transition probability and said observation probability based on said frequency variables; and wherein a first constraint is applied in said learning so that the number of observation symbols corresponding to said observation signal observed at one node of the HMM becomes one, wherein said learning executes processing for finding a node that includes two or more observation symbols to realize said first constraint, and dividing the found node.