1. A method of collecting new words for addition to a lexicon for an agglutinative language, the method comprising: using a processor to retrieve sentences in the agglutinative language from documents; using the processor to identify new word candidate character strings in the retrieved sentences having a predetermined range of number characters; using the processor to filter the identified new word candidate character strings using a combination of a plurality of statistical criteria to generate a new words list, the plurality of statistical criteria comprising: a frequency criteria comprising a frequency of occurrence of a new word candidate character string in the retrieved sentences; a variance criteria comprising a left-hand side variance criteria comprising a number of different single characters which appear adjacent a left-hand side of the new word candidate character string divided by the frequency of occurrence of the new word candidate character string in the retrieved sentences; a character association criteria comprising a frequency of occurrence of two characters appearing adjacent one another in the retrieved sentences, divided by the multiplication product of a frequency of occurrence of a first of the two characters in the retrieved sentences and a frequency of occurrence of a second of the two characters in the retrieved sentences; using the processor to filter the identified new word candidate character strings further comprising: calculating the frequency criteria for all of the new word candidate character strings; calculating the variance criteria for only new word candidate character strings that have a frequency criteria greater than a first threshold; calculating the character association criteria for only new word candidate character strings that have a variance criteria greater than a second threshold; adding a new word candidate character string to the new words list when the character association criteria is greater than a third threshold; and using the processor to add words from the new words list to the lexicon.
1. A method of collecting new words for addition to a lexicon for an agglutinative language, the method comprising: using a processor to retrieve sentences in the agglutinative language from documents; using the processor to identify new word candidate character strings in the retrieved sentences having a predetermined range of number characters; using the processor to filter the identified new word candidate character strings using a combination of a plurality of statistical criteria to generate a new words list, the plurality of statistical criteria comprising: a frequency criteria comprising a frequency of occurrence of a new word candidate character string in the retrieved sentences; a variance criteria comprising a left-hand side variance criteria comprising a number of different single characters which appear adjacent a left-hand side of the new word candidate character string divided by the frequency of occurrence of the new word candidate character string in the retrieved sentences; a character association criteria comprising a frequency of occurrence of two characters appearing adjacent one another in the retrieved sentences, divided by the multiplication product of a frequency of occurrence of a first of the two characters in the retrieved sentences and a frequency of occurrence of a second of the two characters in the retrieved sentences; using the processor to filter the identified new word candidate character strings further comprising: calculating the frequency criteria for all of the new word candidate character strings; calculating the variance criteria for only new word candidate character strings that have a frequency criteria greater than a first threshold; calculating the character association criteria for only new word candidate character strings that have a variance criteria greater than a second threshold; adding a new word candidate character string to the new words list when the character association criteria is greater than a third threshold; and using the processor to add words from the new words list to the lexicon. 2. The method of claim 1 , wherein the step of using the processor to retrieve sentences in the agglutinative language from documents comprises retrieving sentences from web pages.