9. A method as in claim 1 , where identifying tokens to be ignored comprises applying a negative dictionary to the plurality of tokens and wherein the plurality of dictionaries consists of the prefix dictionary, the suffix dictionary, and the negative dictionary.
1. A method to process a document, comprising: partitioning, with a tokenizer operating on at least one computer, document text separated by spaces into a plurality of tokens based on the spaces; identifying, with a token processing unit operating on at least one computer, tokens to be ignored and not considered; determining, with the token processing unit, that a first token considered of the plurality of tokens comprises a chemical name fragment, wherein determining comprises: examining syntax of the first token, examining context of the first token with respect to at least one adjacent token of the plurality of tokens, and taking into account the syntax and the context, applying to the first token a plurality of regular expressions, rules, and a plurality of dictionaries comprised of a prefix dictionary, and a suffix dictionary to recognize the chemical name fragments; adding, with the token processing unit, the recognized chemical name fragment to a vector of chemical name fragments, where the chemical name fragment is identified by a vector index variable; combining, with the token processing unit, the recognized chemical name fragment with at least one of the adjacent tokens that are determined to be a chemical name fragment into a complete chemical name, where combining comprises: initializing the chemical name fragment vector index variable, incrementing the chemical name fragment vector index variable, where the incrementing continues at least until no chemical name fragments remain; setting a string combination to include the chemical name fragments identified by the initialized and incremented chemical name fragment vector index variables, and adding the string combination to a vector c as the complete chemical name; assigning, with a sentence parser unit operating on at least one computer, the complete chemical name with one part of speech; and storing in a memory the complete chemical name assigned with the one part of speech.