1. A method for compressing a grammar, the method comprising: receiving a grammar by using a computer, the grammar comprising a set of rules, each rule comprising a set of token classes, wherein a token class is a logical grouping of tokens, and a token is a string of one or more characters; parsing the grammar to identify the set of rules within the grammar and the set of token classes within each rule; identifying, from the set of token classes within each rule, a set of unimportant token classes separate from a set of important token classes, where the set of unimportant token classes are eligible for compression; analyzing the set of unimportant token classes to identify duplicates within the set of unimportant token classes; merging the set of unimportant token classes to generate a merged token class by removing duplicates within the set of unimportant token classes; and substituting the merged token class in the grammar for the set of unimportant token classes to generate a compressed grammar, wherein substituting the merged token class in the grammar for the set of unimportant token classes that were merged to generate the merged token class comprises substituting the merged token class for all instances within the grammar of the set of unimportant token classes that were merged to generate the merged token class.
1. A method for compressing a grammar, the method comprising: receiving a grammar by using a computer, the grammar comprising a set of rules, each rule comprising a set of token classes, wherein a token class is a logical grouping of tokens, and a token is a string of one or more characters; parsing the grammar to identify the set of rules within the grammar and the set of token classes within each rule; identifying, from the set of token classes within each rule, a set of unimportant token classes separate from a set of important token classes, where the set of unimportant token classes are eligible for compression; analyzing the set of unimportant token classes to identify duplicates within the set of unimportant token classes; merging the set of unimportant token classes to generate a merged token class by removing duplicates within the set of unimportant token classes; and substituting the merged token class in the grammar for the set of unimportant token classes to generate a compressed grammar, wherein substituting the merged token class in the grammar for the set of unimportant token classes that were merged to generate the merged token class comprises substituting the merged token class for all instances within the grammar of the set of unimportant token classes that were merged to generate the merged token class. 6. The method of claim 1 , wherein analyzing the set of unimportant classes comprises employing a similarity function to identify similar unimportant token classes.