1. A computer-implemented method comprising: receiving, by a processor, a string of bytes representing non-segmented text written in a non-delimited language, wherein the non-segmented text has been classified into a predetermined category; simultaneously searching, by the processor, for a plurality of N-grams in a single pass through the string of bytes; collecting, by the processor, statistical information on occurrences of the plurality of N-grams in the string of bytes; generating, by the processor, a model based on the statistical information, the model usable by a content filter to classify contents; defining, by the processor, a plurality of states based on the plurality of N-grams; constructing, by the processor, a finite state machine having the plurality of states, wherein the plurality of states are coupled to each other via one or more paths; and mapping, by the processor, each of the plurality of states to a set of zero or more of the plurality of N-grams in an output table.
1. A computer-implemented method comprising: receiving, by a processor, a string of bytes representing non-segmented text written in a non-delimited language, wherein the non-segmented text has been classified into a predetermined category; simultaneously searching, by the processor, for a plurality of N-grams in a single pass through the string of bytes; collecting, by the processor, statistical information on occurrences of the plurality of N-grams in the string of bytes; generating, by the processor, a model based on the statistical information, the model usable by a content filter to classify contents; defining, by the processor, a plurality of states based on the plurality of N-grams; constructing, by the processor, a finite state machine having the plurality of states, wherein the plurality of states are coupled to each other via one or more paths; and mapping, by the processor, each of the plurality of states to a set of zero or more of the plurality of N-grams in an output table. 2. The method of claim 1 , wherein collecting the statistical information comprises: counting, by the processor, a number of occurrences of each of the plurality of N-grams in the string of bytes.