1. A method of training a randomized decision tree, comprising: training a randomized decision tree through a plurality of tree level iterations, each said tree level iteration expands said randomized decision tree with a plurality of tree nodes of another tree level, each said tree level iteration comprising: receiving a plurality of training data samples, said plurality of training data samples includes a plurality of data subsets, each said data subset corresponds to another of a plurality of attributes; distributing said plurality of data subsets to a plurality of slave processing units after sorting said plurality of data samples in consecutive ascending order by updating a first index that identifies trajectories of said plurality of training data samples through at least one tree node of a previous tree level; simultaneously processing said plurality of data subsets to identify a plurality of split functions with respect to each said data subset for at least one tree node and updating a second index that identifies trajectories of said plurality of training data samples through said at least one tree node of said another tree level; collecting said plurality of split functions from said plurality of slave processing units and constructing said another tree level by selecting a preferred split function for said at least one tree node of said another tree level; and outputting said randomized decision tree by providing said at least one tree node for a plurality of tree levels created through said plurality of tree level iterations; wherein said first index that is used during said another tree level iteration is equal to said second index that is used during said previous tree level iteration.