1. A method of executing a process submitted to a host machine in a distributed computing system, the method comprising the steps of: starting at least first and second executions of the process on respective first and second remote machines in the distributed computing system; utilizing one of the first and second executions of the process to provide an on-demand checkpoint for the other execution of the process in the event the other execution is terminated, such that an additional execution of the process is started from the on-demand checkpoint; and taking a periodic checkpoint of at least one of the first and second executions of the process, the process thereby being protected by a combination of on-demand checkpointing and periodic checkpointing.