1. A computer implemented method for unsupervised stemming schema learning and lexicon acquisition from corpora, the computer implemented method comprising; obtaining a corpus from a corpora; analyzing the corpus to deduce a set of possible stemming schema, wherein analyzing the corpus to deduce the set of possible stemming schema further comprises: generating a first affix count in concepts, wherein each of the concepts is a word unique in the corpus; generating a second affix count in schemas, wherein each of the schemas contains a transformation from a first affix to a second affix for the word; and generating a schema score for the each of the schemas from a combination of the first affix count and the second affix count for the possible stemming schemas to identify useful stemming schemas comprising: identifying a first number of occurrences of the first affix in the corpus for each kernel size; identifying a second number of occurrences of the second affix in the corpus for the each kernel size; identifying a third number of occurrences of the each kernel size in the corpus; and dividing a lesser of the first number of occurrences and the second number of occurrences by the third number of occurrences to form the schema score; reviewing and revising the set of possible stemming schema to create a pruned set of stemming schema; and deducing a lexicon from the corpus using the pruned set of stemming schema.
1. A computer implemented method for unsupervised stemming schema learning and lexicon acquisition from corpora, the computer implemented method comprising; obtaining a corpus from a corpora; analyzing the corpus to deduce a set of possible stemming schema, wherein analyzing the corpus to deduce the set of possible stemming schema further comprises: generating a first affix count in concepts, wherein each of the concepts is a word unique in the corpus; generating a second affix count in schemas, wherein each of the schemas contains a transformation from a first affix to a second affix for the word; and generating a schema score for the each of the schemas from a combination of the first affix count and the second affix count for the possible stemming schemas to identify useful stemming schemas comprising: identifying a first number of occurrences of the first affix in the corpus for each kernel size; identifying a second number of occurrences of the second affix in the corpus for the each kernel size; identifying a third number of occurrences of the each kernel size in the corpus; and dividing a lesser of the first number of occurrences and the second number of occurrences by the third number of occurrences to form the schema score; reviewing and revising the set of possible stemming schema to create a pruned set of stemming schema; and deducing a lexicon from the corpus using the pruned set of stemming schema. 4. The computer implemented method of claim 1 , wherein reviewing and revising the set of possible stemming schema to create a pruned set of stemming schema further comprises: analyzing each stemming schema of the set of possible stemming schema; and identifying stemming schema having a kernel that has a best schema matching a current stemming schema and keeping the kernel, otherwise deleting the stemming schema.