1. A method, implemented on at least one machine each of which has at least one processor, storage, and a communication platform connected to a network for distributed machine learning on a cluster including a plurality of nodes, the method comprising the steps of: performing a machine learning process in each of the plurality of nodes based on a respective subset of training data to calculate a local parameter, wherein the training data is partitioned over the plurality of nodes; determining a plurality of operation nodes from the plurality of nodes based on a status of the machine learning process performed in each of the plurality of nodes; connecting the plurality of operation nodes to form a network topology; and generating an aggregated parameter by merging local parameters calculated in each of the plurality of operation nodes in accordance with the network topology.