1. A computer implemented method for classifying a data stream using high-order models based on an underlying class distribution, the computer implemented method comprising: dividing the data stream into a plurality of data segments using a processor, and wherein the data segments are divided into training data and testing data, wherein the training data is used to determine one of a set of states, wherein the dividing step is performed on a data processing system; learning a classifier for each data segment in the training data using a data mining application on the data processing system, wherein learning the classifier for the each data segment in the training data further comprises: finding a top two classifiers in terms of classification accuracy for each of the plurality of data segments; forming each of the top two classifiers into a set of transactions; and using the data mining application to find all k-frequent itemsets in the set of transactions, wherein each k-frequent itemset corresponds to a set of classifiers clustering the set of classifiers into the set of states; computing a probability of a first state transitioning to a second state from the set of states using a formula P i , j = A ⁡ [ i , j ]  S i  , wherein the probability is known as P i,j , and wherein using the formula P i , j = A ⁡ [ i , j ]  S i  further comprises: counting a number of cases where a first state known as S i is followed by state S j in a state sequence, wherein the state sequence corresponds to a given historical data sequence; and storing the number of the cases when the first state known as S i is followed by state S j in a two dimensional array, wherein the two dimensional array is known as A[i,j]; creating a state transition diagram, wherein the state transition diagram is a high order model corresponding to the set of states, wherein the set of states capture the underlying class distribution, wherein the state transition diagram represents a probabilistic transition of data in the data stream from one state to another state from among the set of states, wherein the state transition diagram is comprised of nodes, wherein the state transition diagram further comprises connecting edges between each node, wherein the connecting edges are transitional probabilities between the set of states, wherein the transitional probabilities are calculated using the formula P i , j = A ⁡ [ i , j ]  S i  ; ⁢ and using the state transition diagram to classify the testing data from the data stream into corresponding states from the set of states.