47. An apparatus for highlighting and categorizing images from a document using a sequence of word tokens representing words of the document, the word tokens comprising character shape code classes, each word of the document being represented by only one word token, the apparatus comprising: means for eliminating predetermined character shape code classes from said sequence of word tokens; means for removing predetermined common function word tokens and numerical word tokens from said sequence of word tokens to form a reduced sequence of word tokens using a pattern matching technique and a stop token list comprising an optional token list; means for determining word token frequency appearance rates for the word tokens of the reduced sequence; means for ranking said frequency of appearance rates; means for determining the nth or more most frequently appearing word tokens based on the ranked frequency of appearance rates; and means for categorizing the document into one of a plurality of pre-existing categories.
47. An apparatus for highlighting and categorizing images from a document using a sequence of word tokens representing words of the document, the word tokens comprising character shape code classes, each word of the document being represented by only one word token, the apparatus comprising: means for eliminating predetermined character shape code classes from said sequence of word tokens; means for removing predetermined common function word tokens and numerical word tokens from said sequence of word tokens to form a reduced sequence of word tokens using a pattern matching technique and a stop token list comprising an optional token list; means for determining word token frequency appearance rates for the word tokens of the reduced sequence; means for ranking said frequency of appearance rates; means for determining the nth or more most frequently appearing word tokens based on the ranked frequency of appearance rates; and means for categorizing the document into one of a plurality of pre-existing categories. 48. The apparatus according to claim 47 wherein the means for categorizing comprises: means for determining a token profile for said document; means for measuring at least one distance between the token profile and at least one reference profile developed from reference data; and means for categorizing the document into at least one of said pre-existing categories based on the at least one distance measured.