1. A method of processing a sequence of tokens to detect predetermined data, comprising: receiving the sequence of tokens, wherein each token has at least one token type and the predetermined data has a structure that comprises at least one required token of first predetermined token type and further includes at least one other optional token of a second predetermined token type; comparing, by a processor in a starting state of a decision tree, a token type of a first token in the sequence with one or more conditions provided for the starting state; if one of the starting state conditions is not met, determining the predetermined data cannot be detected; and if one of said starting state conditions is met, updating the sequence by consuming the first token and proceeding to a next state of the decision tree corresponding to the met condition, if there is a token remaining in the updated sequence, comparing the token type of the token in the updated sequence with one or more conditions provided for the now current state, if one of said next state conditions is met, updating the sequence by consuming the token and proceeding to the next state corresponding to the met condition, iterating said comparing of the updated sequence and said updating of the sequence until there is no token remaining in the sequence matching a condition of the current state, determining whether a reduction is associated with the current state and, if so, using the reduction, and if no reduction is associated with the current state, determining that the predetermined data cannot be detected.
1. A method of processing a sequence of tokens to detect predetermined data, comprising: receiving the sequence of tokens, wherein each token has at least one token type and the predetermined data has a structure that comprises at least one required token of first predetermined token type and further includes at least one other optional token of a second predetermined token type; comparing, by a processor in a starting state of a decision tree, a token type of a first token in the sequence with one or more conditions provided for the starting state; if one of the starting state conditions is not met, determining the predetermined data cannot be detected; and if one of said starting state conditions is met, updating the sequence by consuming the first token and proceeding to a next state of the decision tree corresponding to the met condition, if there is a token remaining in the updated sequence, comparing the token type of the token in the updated sequence with one or more conditions provided for the now current state, if one of said next state conditions is met, updating the sequence by consuming the token and proceeding to the next state corresponding to the met condition, iterating said comparing of the updated sequence and said updating of the sequence until there is no token remaining in the sequence matching a condition of the current state, determining whether a reduction is associated with the current state and, if so, using the reduction, and if no reduction is associated with the current state, determining that the predetermined data cannot be detected. 2. A method according to claim 1 , wherein at least one of one or more reductions associated with one or more states of the decision tree determines that the predetermined data has been detected.