1. A method, operative in a cluster of nodes, the cluster comprising a first node, and a set of second nodes, each node in the cluster comprising a computing entity having a hardware processor and a memory, the method comprising: partitioning a dataset comprising blocks of data over the set of second nodes, each second node having a block of the dataset, wherein the block of the dataset associated with a respective second node is larger than a size of the memory; and for each of a set of iterations: at each respective second node, and concurrently for all second nodes, solving an optimization problem with respect to the block of the dataset associated with the respective second node to generate an update; after solving the optimization problem, each respective second node forwarding its update to the first node; at the first node, processing the updates forwarded from the second nodes to generate a result; and forwarding the result from the first node back to each respective second node for use by the second node in a next iteration in the set of iterations; wherein the dataset is a training set for a support vector machine and comprises advertising modeling data.