1. A signal processing device for processing a plurality of observed signals at a plurality of frequencies, the plurality of the observed signals being produced by a plurality of sound receiving devices which receive a mixture of a plurality of sounds, the signal processing device comprising: a storage unit that stores observed data of the plurality of the observed signals, the observed data representing a time series of magnitude of each frequency in each of the plurality of the observed signals; an index calculation unit that calculates an index value from the observed data for each of the plurality of the frequencies, the index value indicating significance of learning of a separation matrix using the observed data of each frequency of the plurality of the frequencies and representing a total number of bases that are axis lines of regions in which observed vectors obtained from the observed data are distributed, each observed vector including, as elements, respective magnitudes of a corresponding frequency in the plurality of the observed signals, the separation matrix being used for separation of the plurality of the sounds; a frequency selection unit that selects at least one frequency from the plurality of the frequencies according to the index value of each frequency calculated by the index calculation unit by selecting one or more frequencies at which the total number of the bases represented by the index value is larger than a total number of bases represented by index values at other frequencies; and a learning processing unit that determines the separation matrix by learning with a given initial separation matrix using the observed data of the frequency selected by the frequency selection unit among the plurality of the observed data stored in the storage unit.