1. A method of selecting data for an automated training process comprising: identifying a plurality of occurrences of a non-standard token in a text corpus stored in a memory; identifying a first plurality of tokens in the text corpus that are located proximate to at least one of the occurrences of the non-standard token; identifying a plurality of occurrences of a candidate standard token in the text corpus; identifying a second plurality of tokens in the text corpus that are located proximate to at least one of the occurrences of the candidate standard token; identifying a contextual similarity between the first plurality of tokens and the second plurality of tokens; generating a statistical model for correction of non-standard tokens with the non-standard token in association with the standard token for generation of a statistical model only in response to the identified contextual similarity being greater than a predetermined threshold; and storing the generated statistical model in the memory for use in identification of another standard token that corresponds to another non-standard token identified in text data that are not included in the text corpus.
1. A method of selecting data for an automated training process comprising: identifying a plurality of occurrences of a non-standard token in a text corpus stored in a memory; identifying a first plurality of tokens in the text corpus that are located proximate to at least one of the occurrences of the non-standard token; identifying a plurality of occurrences of a candidate standard token in the text corpus; identifying a second plurality of tokens in the text corpus that are located proximate to at least one of the occurrences of the candidate standard token; identifying a contextual similarity between the first plurality of tokens and the second plurality of tokens; generating a statistical model for correction of non-standard tokens with the non-standard token in association with the standard token for generation of a statistical model only in response to the identified contextual similarity being greater than a predetermined threshold; and storing the generated statistical model in the memory for use in identification of another standard token that corresponds to another non-standard token identified in text data that are not included in the text corpus. 2. The method of claim 1 , the identification of the contextual similarity further comprising: identifying a first weight vector including a first plurality of weights with each weight in the first plurality of weights being an identified weight for one token in the first plurality of tokens; identifying a second weight vector including a second plurality of weights with each weight in the second plurality of weights being an identified weight for one token in the second plurality of tokens; identifying a sum of products of the first plurality of weights in the first weight vector multiplied by corresponding weights in the second plurality of weights in the second weight vector; identifying a first square root of a sum of squared values of the first plurality of weights in the first weight vector; identifying a second square root of a sum of squared values of the second plurality of weights in the second weight vector; and identifying the contextual similarity with reference to the identified sum divided by a product of the first square root multiplied by the second square root.