1. A system for efficiently computing iterative functions over aggregate statistics of a distributed data set, said system comprising: a plurality of processors programmed to perform distributed computations, the distributed data set partitioned into a plurality of data subsets, one for each of the plurality of processors, wherein each of the plurality of processors includes: 1) statistics describing a current best estimate of the aggregate statistics; and 2) a differential vector describing changes to the statistics for a current iteration; wherein each of the plurality of processors is programmed to: iteratively perform a plurality of sweeps over the data subset of the processor to update the statistic until a termination criteria is satisfied; compute a differential vector representing the changes made during the update; save the differential vector of the processor for a predetermined number of past generations, each generation corresponding to an iteration of the processor; receive a generation-specific update from a sending one of the plurality of processors, the generation-specific update including a differential vector of the sending processor corresponding to the specific generation of the generation-specific update; and aggregate the differential vector of the generation-specific update with the statistics of the receiving processor.