1. A computer-implemented method, comprising: for a pre-defined iterative computation: dividing an iteration space over which the iterative computation operates to define two or more items of work that when executed collectively perform the iterative computation; allocating the two or more items of work on respective nodes of a plurality of nodes for execution in parallel, wherein at least two of the two or more items of work are allocated for execution on different nodes; recording an indication of the respective node on which each of the two or more items of work was allocated for execution; executing the iterative computation, wherein said executing comprises executing one of the two or more items of work on a node other than the node on which it was allocated for execution; and updating the indication of the node on which the one of the two or more items of work was allocated for execution to reflect the node on which it is actually executed.